GithubHelp home page GithubHelp logo

kdcllc / cometd.netcore.salesforce Goto Github PK

View Code? Open in Web Editor NEW
45.0 6.0 24.0 294 KB

CometD Salesforce Implementation.

License: MIT License

C# 99.87% Shell 0.13%
salesforce salesforce-developers salesforce-api salesforce-rest-api bay dotnet-cli dotnet-core salesforce-apex cometd workbench

cometd.netcore.salesforce's Introduction

CometD.NetCore.Salesforce

GitHub license Build status NuGet Nuget feedz.io

Note: Pre-release packages are distributed via feedz.io.

Summary

This repo contains the CometD .NET Core implementation for Salesforce Platform events.

These events can be subscribed to and listened to by your custom Event Listener. The sample application of this library can be found here.

The solution contains the following:

  1. CometD.NetCore2.Salesforce

  2. DotNet Cli tool salesforce

buymeacoffee

Give a Star! โญ

If you like or are using this project to learn or start your solution, please give it a star. Thanks!

Install

    dotnet add package CometD.NetCore.Salesforce

Saleforce Setup

Watch Video

  1. Sing up for development sandbox with Saleforce: https://developer.salesforce.com/signup.
  2. Create Connected App in Salesforce.
  3. Create a Platform Event.

Create Connected App in Salesforce

  1. Setup -> Quick Find -> manage -> App Manager -> New Connected App.
  2. Basic Info:

info

  1. API (Enable OAuth Settings): settings

  2. Retrieve Consumer Key and Consumer Secret to be used within the Test App

Create a Platform Event

  1. Setup -> Quick Find -> Events -> Platform Events -> New Platform Event:

event

  1. Add Custom Field

event

(note: use sandbox custom domain for the login to workbench in order to install this app within your production)

Use workbench to test the Event workbench

AuthApp

OAuth Refresh Token Flow

Use login instead of test Simple application that provides with Web Server OAuth Authentication Flow to retrieve Access Token and Refresh Token to be used within the application.

Username/Password Flow

To enable Username/Password flow and grant type, simply omit the auth token and refresh token while providing the username, password and user api token.

Special thanks to our contributors

Related projects

cometd.netcore.salesforce's People

Contributors

apaulro avatar cternes avatar jolienai avatar kdcllc avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

cometd.netcore.salesforce's Issues

Not receiving new custom messages from Salesforce after a handshake has taken place

Network connection loss prevents receipt of new messages.

When a network connection is lost, the expected behavior is that the client will reestablish the connection with Salesforce and run the method added to client.reconnect.

The actual behavior is that when a network connection has been lost the client reconnects with Salesforce and performs a successful handshake, but does not run the method added client.reconnect.

To resolve the issue about reconnect event not being invoked, please see pull request #13 for a possible solution.

Edit: Added clarity

Disconnect() method is giving Error.

I am receiving below error when trying to call BayeuxClient.Disconnect(). It was working earlier for me but now it has stopped working and I am not able to disconnect. Please look into this issue.
Error Message: Recursive read lock acquisitions not allowed in this mode.
Stack Trace:
at System.Threading.ReaderWriterLockSlim.TryEnterReadLockCore(TimeoutTracker timeout)
at System.Collections.Generic.ThreadSafeList1.<GetEnumerator>d__18.MoveNext() in C:\projects\cometd-netcore\src\CometD.NetCore\Internal\ThreadSafeList.cs:line 201 at CometD.NetCore.Common.AbstractClientSession.ExtendSend(IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Common\AbstractClientSession.cs:line 214 at CometD.NetCore.Client.BayeuxClient.BayeuxClientState.Send(ITransportListener listener, IList1 messages, Int32 clientTimeout) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 1003
at CometD.NetCore.Client.BayeuxClient.BayeuxClientState.Send(ITransportListener listener, IMutableMessage message, Int32 clientTimeout) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 987
at CometD.NetCore.Client.BayeuxClient.DisconnectingState.Execute() in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 1245
at CometD.NetCore.Client.BayeuxClient.UpdateBayeuxClientState(BayeuxClientStateUpdater_createDelegate create, BayeuxClientStateUpdater_postCreateDelegate postCreate) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 727
at CometD.NetCore.Client.BayeuxClient.UpdateBayeuxClientState(BayeuxClientStateUpdater_createDelegate create) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 690
at CometD.NetCore.Client.BayeuxClient.Disconnect() in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 143
at CometD.NetCore.Common.AbstractClientSession.ExtendReceive(IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Common\AbstractClientSession.cs:line 188
at CometD.NetCore.Common.AbstractClientSession.Receive(IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Common\AbstractClientSession.cs:line 155
at CometD.NetCore.Client.BayeuxClient.ProcessMessage(IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 609
at CometD.NetCore.Client.BayeuxClient.PublishTransportListener.ProcessMessage(IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 796
at CometD.NetCore.Client.BayeuxClient.PublishTransportListener.OnMessages(IList`1 messages) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 767
at CometD.NetCore.Client.Transport.LongPollingTransport.GetResponseCallback(IAsyncResult asynchronousResult) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\Transport\LongPollingTransport.cs:line 311
Source: System.Private.CoreLib

Subscribing to multiple (3+) push topics results in an Error

When I subscribe to 1 or 2 push topics, everything is fine. As soon as I add a 3rd subscription, it will connect to each topic successfully, then disconnect and set workers to 0. It appears to be some type of race condition in the LongPollingTransport GetResponseCallback(IAsyncResult asynchronousResult) method, where cookies are getting added to the exchange at the same time that the collection is being iterated over.

ComentD Event Replay?

Hello, I'm currently working on a process that will subscribe to a platform event. I would like to use the replay feature and possibly get the messages that the process may have missed if stopped for any reason. How is that implemented with this library?

Add support for handling 401 Authentication Errors from Salesforce

Per the section noted here, this client needs to support the scenario when BayeuxClient receives an error value of 401::Authentication invalid :
https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/streaming_handling_errors.htm

401 Authentication Errors

Client authentication can sometimes become invalid, for example, when the OAuth token is revoked or a Salesforce admin revokes the Salesforce session. An admin can revoke an OAuth token or delete a Salesforce session to prevent a client from receiving events. Sometimes a client can inadvertently invalidate its authentication by logging out from a Salesforce session. Streaming API regularly validates the OAuth token or session ID while the client is connected. If client authentication is not valid, the client is notified with an error. A Bayeux message is sent on the /meta/connect channel with an error value of 401::Authentication invalid and an advice field containing reconnect=none. After receiving the error notification in the channel listener, the client must reauthenticate and reconnect to receive new events.

Silently fails when replay id is too old.

I created a listener that grabs the latest received replayId from a separate table and reconnects using that particular replayId for continuity. The listener stayed alive for 2 days with no errors, but must of had a stale connection of sorts, as new messages stopped coming in. After realizing that the messages were being missed, I checked the PushTopic on Workbench and entered the latest replayId the listener was trying.

Workbench Error:
7. Subscription Failure: 400::The replayId {82} you provided was invalid. Please provide a valid ID, -2 to replay all events, or -1 to replay only new events.

"error": "400::The replayId {82} you provided was invalid. Please provide a valid ID, -2 to replay all events, or -1 to replay only new events.",
"successful": false

The CometD client knew nothing of this error and just pretended that the connection was all fine and dandy. As a workaround for now, I am now restarting my listener every hour to make sure that the connection is fresh.

It would make sense to me that if this error actually was received and an exception thrown, I could handle it with a new connection asking for -2 and doing a reconciliation of duplicates as they come in.

Replaying historical events?

Hi,

Does this support replaying historical push topic notifications?

When our service starts up, I've subscribed to an existing push topic "/topic/SomeTopicName" (i've used an example here) using the following code

IClientSessionChannel channel = _bayeuxClient.GetChannel("/topic/SomeTopicName", 102);
channel?.Subscribe(listener);

When the application starts up, i was expecting our subscribers to pickup historic notifications starting with replayId of 102 for the "/topic/SomeTopicName" - this was not happening. it was only picking up new events.

Subscribing with the last replay id makes the app receive the older events

The following issue occurs when a session is closed after 3 hours of inactivity. We are subscribing with the last replay id but the app receives older (already received) events.

code in main()

bayeuxClient.AddExtension(new ReplayExtension());
ClientSessionChannelListener clientSessionChannel = new ClientSessionChannelListener();
bayeuxClient.GetChannel(ChannelFields.META_HANDSHAKE).AddListener(clientSessionChannel);

code in ClientSessionChannelListener


public class ClientSessionChannelListener : IMessageListener
{
        public void OnMessage(IClientSessionChannel channel, IMessage message)
        {
            long lastProcessedReplayId = GetLastProcessedReplayId();
            _bayeuxClient.GetChannel('/topic', lastProcessedReplayId).Subscribe(new Listener(_platformEventsMessage, channelInfo));                        
        }
}

If we do not pass the last id, the app does not receive the already processed events (the issue does not occur).

_bayeuxClient.GetChannel('/topic').Subscribe(new Listener(_platformEventsMessage, channelInfo));

Unable to Use the Salesforce Test Tenant

I would like to be able to use the Salesforce test tenant with this library.

When using the following configuration:
"LoginUrl": "https://test.salesforce.com",
you get an error when the token refresh method is called inside the library. This is because the configured LoginUrl is not used when making the call:

await authClient.TokenRefreshAsync(
                           options.RefreshToken,
                           options.ClientId)

This is fixed in PR #9

Error connecting to salesforce

I am getting the following Error

warn: CometD.NetCore.Salesforce.Resilience.ResilientForceClient[0]
CountQueryAsync wait 2.005 to execute with exception: One or more errors occurred. (expired access/refresh token) for named policy: ResilientForceClientWaitAndRetryAsync
warn: CometD.NetCore.Salesforce.Resilience.ResilientForceClient[0]
CountQueryAsync wait 4.055 to execute with exception: One or more errors occurred. (expired access/refresh token) for named policy: ResilientForceClientWaitAndRetryAsync
warn: CometD.NetCore.Salesforce.Resilience.ResilientForceClient[0]
CountQueryAsync wait 8.099 to execute with exception: One or more errors occurred. (expired access/refresh token) for named policy: ResilientForceClientWaitAndRetryAsync

"Salesforce": {
"ClientId": "3MVG...............is_L3Bmr",
"ClientSecret": "8B.............................C8E588",
"RefreshToken": "00D7b00.........................xMPT7ohNoA5RLxUbK0dbxPb4gbSDQDTf8ykvL",
"AccessToken": "00D7b00000....................oA5RLxUbK0dbxPb4gbSDQDTf8ykvL",
"RedirectUri": "http://localhost:5050/",
"OrganizationUrl": "",
"LoginUrl": "https://XXXX--migrate.my.salesforce.com",
"OAuthUri": "/services/oauth2/token",
"PublishEndpoint": "/services/data/v42.0/sobjects/",
"EventOrTopicUri": "/event",
"CometDUri": "/cometd/42.0",
"Retry": 3,
"BackoffPower": 2,
"CustomEvent": "Custom_Event__e",
"ReplayId": "-2"
}

[AuthApp] An error occurred while writing to logger(s)

Hi there. Thanks for this great repo. Hopefully it'll do exactly what I want.
At the moment though, I am stuck trying to use the AuthApp tool.
If I add an appsettings.json and compile the project, then run it from the bin folder, I get:
An error occurred while writing to logger(s). (Index (zero based) must be greater than or equal to zero and less than the size of the argument list.)
Same if I F5 it in VS 2019.

The (redacted) command line I'm running (from a vs code ps terminal) is:

.\AuthApp.exe get-tokens --key:3M... ...r7 --secret:A8... ...CA --verbose:debug --section:Salesforce

appsettings.json (redacted) looks like:

{
  "AzureVault": {
    "BaseUrl": null
  },
  "Salesforce": {
    "ClientId": "3M... ....r7",
    "ClientSecret": "A8... ...CA",
    "RefreshToken": "",
    "AccessToken": "",
    "RedirectUri": "http://localhost:5050/",
    "OrganizationUrl": "",
    "LoginUrl": "https://login.salesforce.com",
    "OAuthUri": "/services/oauth2/token",
    "PublishEndpoint": "/services/data/v42.0/sobjects/",
    "EventOrTopicUri": "/event",
    "CometDUri": "/cometd/42.0",
    "Retry": 2,
    "CustomEvent": "Custom_Event__e",
    "ReplayId": "-2"
  }
}

Top half of the stacktrace:

   at Microsoft.Extensions.Logging.Logger.ThrowLoggingError(List`1 exceptions)
   at Microsoft.Extensions.Logging.Logger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter)
   at Microsoft.Extensions.Logging.Logger`1.Microsoft.Extensions.Logging.ILogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter)
   at Microsoft.Extensions.Logging.LoggerExtensions.Log(ILogger logger, LogLevel logLevel, EventId eventId, Exception exception, String message, Object[] args)
   at Microsoft.Extensions.Hosting.HostStartupService.StartAsync(CancellationToken cancellationToken) in C:\projects\bet-aspnetcore\src\Bet.Extensions.Hosting\HostStartupService.cs:line 43
   at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>d__9.MoveNext()

I have security issues trying to run the "dotnet tool install" command which is why Im using the source code.

Any help greatly appreciated

Not receiving events from salesforce after 1-2 hrs of inactivity.

When I am running the console application which I have configured to listen to a Push topic its running fine. But when there is no messages or activity then after 1-2 hours no message is received even if events occurred on salesforce side.

Is there some that needs to be done may be some configuration or anything ?

System.NullReferenceException: Object reference not set to an instance of an object in CometD.NetCore.Salesforce.ResilientStreamingClient.ErrorExtension_ConnectionError function.

Currently, we are using this library to listen to push topics whenever there is a change in salesforce. We are using this library as follows.

Code for registering an event and configuring the Salesforce Streaming Client in Program.cs is:

public static void Main(string[] args)
{
var builder = WebApplication.CreateBuilder(args);
builder.Host.ConfigureServices((hostContext, services) =>
{
var salesforceConfiguration = services.BuildServiceProvider().GetRequiredService<IOptions>().Value;
services.AddResilientStreamingClient("", "", (conf) =>
{
conf.ClientId = salesforceConfiguration.ClientId;
conf.ClientSecret = salesforceConfiguration.ClientSecret;
conf.RefreshToken = salesforceConfiguration.RefreshToken;
conf.LoginUrl = salesforceConfiguration.BaseUrl;
conf.OAuthUri = Constants.OAuthUriConfig;
conf.EventOrTopicUri = Constants.EventOrTopicConfig;
conf.CometDUri = salesforceConfiguration.CometdUrl;
});
services.AddSingleton<IEventBus, EventBus>();
services.AddHostedService();
services.AddTransient<IMessageListener, UpdatedListener>();
services.AddTransient<IMessageListener, DeletedListener>();
});

    var app = builder.Build();
    app.Run();
}

Code for subscribing to events is:

public class SalesforceEventBusHostedService:IHostedService
{
private readonly ILogger _logger;
private readonly IEventBus _eventBus;
private readonly ICacheService _cacheService;

//Make a list of push topics that we created in salesforce
private readonly List<Tuple<string,int,Type>> _eventsMapping = new()
{
new Tuple<string,int,Type>("topic/Updated",-1,typeof(UpdatedListener)),
new Tuple<string, int, Type>("topic/Deleted", -1, typeof(DeletedListener))
};

public SalesforceEventBusHostedService(ILogger<SalesforceEventBusHostedService> logger,IEventBus eventBus, ICacheService<ReplayIdDto> cacheService)
{
    _logger = logger;
    _eventBus = eventBus;
    _cacheService = cacheService;
    _cacheService.CacheRegion = $"{Constants.MessageListenerCacheRegion}";
}

private static object[] GetPlatformEventObject(string eventName,int replayId,Type eventType)
{
    var platformEvent = Activator.CreateInstance(typeof(PlatformEvent<>).MakeGenericType(eventType));
    platformEvent?.GetType().GetProperty("Name")?.SetValue(platformEvent, eventName);
    platformEvent?.GetType().GetProperty("ReplayId")?.SetValue(platformEvent, replayId);
    return new[] {platformEvent};
}

private async Task SubscribeToEvent(string eventName,int replayId,Type eventType)
{
    var platformEvent = GetPlatformEventObject(eventName, replayId, eventType);
    var subscribeMethod = typeof(IEventBus).GetMethod("Subscribe")?.MakeGenericMethod(eventType);
    await ((Task) subscribeMethod?.Invoke(_eventBus, platformEvent))!;
}

private async Task UnSubscribeToEvent(string eventName,int replayId,Type eventType)
{
    var platformEvent = GetPlatformEventObject(eventName, replayId, eventType);
    var subscribeMethod = typeof(IEventBus).GetMethod("Unsubscribe")?.MakeGenericMethod(eventType);
    await ((Task) subscribeMethod?.Invoke(_eventBus, platformEvent))!;
}

public async Task StartAsync(CancellationToken cancellationToken)
{
    _logger.LogInformation($"{nameof(SalesforceEventBusHostedService)} starting.");
    for (var i = 0; i < _eventsMapping.ToList().Count; i++)
    {
        var (originalEventName, originalReplayId, originalEventType) = _eventsMapping[i];
        var replayIdFromCache = (await _cacheService.GetItemAsync(StreamingApiHelper.RemoveEventOrTopicPrefix(originalEventName)))?.ReplayId??originalReplayId;
        var (eventName, replayId, eventType) = _eventsMapping[i] = new Tuple<string, int, Type>(originalEventName, replayIdFromCache, originalEventType);
        await SubscribeToEvent(eventName, replayId, eventType);
    }
}

public async Task StopAsync(CancellationToken cancellationToken)
{
    _logger.LogInformation($"{nameof(SalesforceEventBusHostedService)} stopped.");
    foreach (var (eventName,replayId, eventType) in _eventsMapping)
    {
        await UnSubscribeToEvent(eventName, replayId, eventType);
    }
}

}

But I face an issue like:

System.NullReferenceException: Object reference not set to an instance of an object.
at CometD.NetCore.Salesforce.ResilientStreamingClient.ErrorExtension_ConnectionError(Object sender, String e) in C:\projects\cometd-netcore-salesforce\src\CometD.NetCore.Salesforce\ResilientStreamingClient.cs:line 210
at CometD.NetCore.Client.Extension.ErrorExtension.ReceiveMeta(IClientSession session, IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\Extension\ErrorExtension.cs:line 74
at CometD.NetCore.Common.AbstractClientSession.ExtendReceive(IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Common\AbstractClientSession.cs:line 188
at CometD.NetCore.Common.AbstractClientSession.Receive(IMutableMessage message) in C:\projects\cometd-netcore\src\CometD.NetCore\Common\AbstractClientSession.cs:line 155
at CometD.NetCore.Client.BayeuxClient.PublishTransportListener.OnMessages(IList`1 messages) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\BayeuxClient.cs:line 769
at CometD.NetCore.Client.Transport.LongPollingTransport.GetResponseCallback(IAsyncResult asynchronousResult) in C:\projects\cometd-netcore\src\CometD.NetCore\Client\Transport\LongPollingTransport.cs:line 310

Can anyone please guide me on why this error is received and if there is something I should be updating on our end?

Need help getting a simple example running.

I'm trying to use the code in this repo to follow this tutorial:
https://developer.salesforce.com/docs/atlas.en-us.api_streaming.meta/api_streaming/code_sample_java_create_pushtopic.htm

But I am at a loss on how to exactly use the library.
I have the streamingclient and I do a subscribetopic. But nothing happens. The state on the client seems to be disconnected.
I expected to see some events happening. But I cannot find anything to make it connect. There seems to be a disconnect function but not a connect function.

Any help appreciated.

cobbled together stuff
`
var host = new HostBuilder()
.ConfigureHostConfiguration(configHost =>
{
configHost.SetBasePath(Directory.GetCurrentDirectory());
configHost.AddJsonFile("hostsettings.json", optional: true);
configHost.AddEnvironmentVariables(prefix: "TESTAPP_");
configHost.AddCommandLine(args);
})
.ConfigureAppConfiguration((hostContext, configBuilder) =>
{
configBuilder.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true);
configBuilder.AddJsonFile(
$"appsettings.{hostContext.HostingEnvironment.EnvironmentName}.json",
optional: true);

                 //configBuilder.AddAzureKeyVault(hostingEnviromentName: hostContext.HostingEnvironment.EnvironmentName);

                 configBuilder.AddEnvironmentVariables(prefix: "TESTAPP_");
                 configBuilder.AddCommandLine(args);

                 if (hostContext.HostingEnvironment.IsDevelopment())
                 {
                     // print out the environment
                     var config = configBuilder.Build();
                     //config.DebugConfigurations();
                 }
             })
             .ConfigureServices((context, services) =>
             {
                 services.AddResilientStreamingClient("Salesforce");

               
                 // Conjure up a RequestServices
                 services.AddTransient<IServiceProviderFactory<IServiceCollection>, DefaultServiceProviderFactory>();
             })
             .ConfigureLogging((hostContext, configLogging) =>
             {
                 configLogging.AddConfiguration(hostContext.Configuration.GetSection("Logging"));
                 configLogging.AddConsole();
                 configLogging.AddDebug();
             })
             .UseConsoleLifetime()
             .Build();

        var sp = host.Services;

        IStreamingClient x = sp.GetRequiredService<IStreamingClient>();
        
        x.SubscribeTopic("/InvoiceStatementUpdates", new MyListener(), -2);
        

        await host.RunAsync();

`

Upgrade the library

  1. Upgrade to the latest dotnetcore 3.1

  2. Remove Obsoletes

  • IAuthenticationClientProxy
  • IForceClientProxy
  • IAuthenticationClientProxy
  • IForceClientProxy
  • StreamingClient

ReplayId - getting older messages after a period of time where channel was subscribed using the last successful replayId

We are having an issue with setting the replayId at the time a channel subscription is re-subscribed.

Here is what we do to enable setting the replayid on subscribing the channel. We setup the Replay Extension at setup time (excerpt):
var bayeuxClient = new BayeuxClient(endpoint, new[] {transport}); bayeuxClient.AddExtension(new ReplayExtension());

later when subscribing the channel the replayId is set for getting new messages:
public void Subscribe(BayeuxClient bayeuxClient) { long replayId = GetLastSuccessfulReplayId(); //pulling last successfull replayid for channel from database Channel = bayeuxClient.GetChannel(ChannelName, replayId); Channel.Subscribe(this); }

If we are re-subscribing the channel due to a connection issue, the handshake method is received and the subscription with an update replayid above has no effect - it restarts the subscription from the first time it subscribed.

We need a way to SET the replayId in the case of a re-subscription, but the ReplayId is read only.

In summary, we start up the Bayeux client, add listeners to all of the meta channels, and perform a Handshake on Bayeux client. On the handshake meta channel listener, we detect a successful handshake and in turn loop through a collection of channel listeners and have them self-subscribe to the channel based on the last successfully processed replayid (stored in database per channel).

The general idea we are following is that when subscribing, we use the last successful replayid stored in the database and when processing an event we store the replayid for the channel in the database. It seems that handshake event happens several times during the day perhaps based on what the client is detecting as an issue with the connection. Is that normal? It seems to be after one of these multiple self-initiated handshakes the setting of the replayid on subscription is ignored; it is only used when subscribing for the first time. If anyone has any ideas or experiences it would be appreciated!

Use AsynExpiringLazy from Bet.Extensions library

            var instanceClient = new AsyncExpiringLazy<ForceClient>(async data =>
            {
                if (data.Result == null
                || DateTime.UtcNow > data.ValidUntil.Subtract(TimeSpan.FromSeconds(5)))
                {
                    var authClient = new AuthenticationClient();

                    await authClient.TokenRefreshAsync(RefreshToken, ClientId);
                    var client = new ForceClient(authClient.AccessInfo.InstanceUrl, authClient.ApiVersion, authClient.AccessInfo.AccessToken);

                    return new AsyncExpirationValue<ForceClient>
                    {
                        Result = client,
                        ValidUntil = DateTimeOffset.UtcNow.AddSeconds(10)
                    };
                }

                return data;
            });

            var client = instanceClient.Value().Result;

Part of this update is to introduce a new way of creating ForceClient with Resilience space.

ResilientStreamingClient is prefered now over StreamingClient

Run locally

There is any way to use this lib locally? I want to do some integration tests in my project.

Thanks in advance

AuthApp enable better errors

Some clean up and enablement of the token generation.

  • Factor out the OAuth2 configurations to SfConfig.cs
  • Standardize switches
  • Enable --verbose:debug switch

cometD Listener does not subscribe to channel after 403 Unknown Client Error

My implementation of cometD.NetCore.Salesforce stops receiving any messaged from Salesforce channel I have subscribed to.
My observations from network are as follows:

This is the communication Pattern:
Initial call sequence
/handshake
/connect
/subscribe

This is followed by a series of /connect loops, each lasting a fixed two minutes or so
About a hundred calls later /connect gets an advice with 403: unknown client
[{"advice":{"interval":0,"reconnect":"handshake"},"channel":"/meta/connect","id":"304","error":"403::Unknown client","successful":false}]

This reponse also has an additional header, not present in earlier responses
Cookies/Login
Set-Cookie: sfdc-stream=!//QiKCtgRz/8Ow+NBSWBTDZ6st4SZ0zgy/W9DuaUlImW4TVx0EbyUQxwr1yj6cye4+kYtTUiqXtxw=; path=/; Expires=Sun, 24-Sep-2023 19:55:37 GMT; SameSite=None; Secure

cometD then initiates /handshake and gets this reponse
[{"ext":{"replay":true,"payload.format":true},"minimumVersion":"1.0","clientId":"1cyu4thyxh3fgv1nqwcvebt3o9x","supportedConnectionTypes":["long-polling"],"channel":"/meta/handshake","id":"305","version":"1.0","successful":true}]

Now cometD resumes /connect calls - the first response is unique, it has an advice
[{"clientId":"1cyu4thyxh3fgv1nqwcvebt3o9x","advice":{"interval":0,"timeout":110000,"reconnect":"retry"},"channel":"/meta/connect","id":"306","successful":true}]

HOWEVER, at this stage there is no message received on the channel from Salesforce.

Just to be clear I am using Bayeux client, not the ResilientStreamingClient.
I have added error listeners but do not receive any messages in those listeners

It appears to me that after /handshake cometD is not resubscribing to the channel.
I don't see this particular issue reported here. Is it a solved problem? Then I would love to know the solution.

Client stopped, unable to find root cause

Hello :)

After a few months of use in a webservice (WebJob) in Azure, I experienced that one of the instances of ResilientStreamingClient stopped (I use 4 clients for 4 separate listeners; 3 kept working and 1 stopped).

The issue is that I am unable to find any cause as to why it suddenly stopped.

I have made some changes to the original source code.

    /// <summary>
    /// Public only for testing purposes
    /// </summary>
    /// <param name="sender"></param>
    /// <param name="e"></param>
    public virtual void ErrorExtension_ConnectionError(
        object? sender,
        string e)
    {
        replayIdToUseOnReconnect = KnownInvalidReplayId;
        _logger.LogError("{name} failed with the following message: {message}", nameof(ResilientStreamingClient), e);
        // authentication failure
        if (string.Equals(e, "403::Handshake denied", StringComparison.OrdinalIgnoreCase)
            || string.Equals(e, "403:denied_by_security_policy:create_denied", StringComparison.OrdinalIgnoreCase)
            || string.Equals(e, "403::unknown client", StringComparison.OrdinalIgnoreCase)
            || string.Equals(e, "401::Authentication invalid", StringComparison.OrdinalIgnoreCase))
        {
            _logger.LogWarning("Handled CometD Exception: {message}", e);

            ReconnectAfterFailure();
        }
        else if (e.Contains("you provided was invalid"))
        {
            _logger.LogError("{name} failed with the following message: {message}", nameof(ResilientStreamingClient), e);
            var start = e.IndexOf('{');
            var end = e.IndexOf('}');
            var replayIdString = e.Substring(start + 1, end - (start + 1));

            if (int.TryParse(replayIdString, out var replayId))
            {
                InvalidReplayIdStrategy(replayId);
                ReconnectAfterFailure();
            }
            else
            {
                _logger.LogCritical("Unable to parse invalid replayId. Unrecoverable without manual intervention");
            }
        }
        else
        {
            _logger.LogError("{name} failed with the following message: {message}", nameof(ResilientStreamingClient), e);
        }
    }

    private void ReconnectAfterFailure()
    {
        // 1. Disconnect existing client.
        Disconnect();

        // 2. Invalidate the access token.
        lock (_tokenLock)
        {
            _tokenResponse.AccessToken = "";
            _tokenResponse.ExpiresAt = DateTimeOffset.MinValue;
        }

        _logger.LogDebug("Invalidate token for {name} ...", nameof(BayeuxClient));

        // 3. Recreate BayeuxClient and populate it with a new transport with new security headers.
        CreateBayeuxClient();

        // 4. Invoke the Reconnect Event
        Reconnect?.Invoke(this, true);
    }

    protected virtual void ErrorExtension_ConnectionException(
        object? sender,
        Exception ex)
    {
        // ongoing time out issue not to be considered as error in the log.
        if (ex?.Message == "The operation has timed out.")
        {
            _logger.LogDebug(ex.Message);
        }
        else if (ex != null)
        {
            _logger.LogError(ex.ToString());
        }
    }

    protected virtual void ErrorExtension_ConnectionMessage(
        object? sender,
        string message)
    {
        _logger.LogDebug(message);
    }

    private void CreateBayeuxClient()
    {
        if (_isDisposed)
        {
            throw new ObjectDisposedException("Cannot create connection when disposed");
        }

        _logger.LogDebug("Creating {name} ...", nameof(BayeuxClient));

        lock(_tokenLock)
        {
            if(_tokenResponse.ExpiresAt < DateTimeOffset.Now || string.IsNullOrWhiteSpace(_tokenResponse.AccessToken))
            {
                _tokenResponse = _authenticator.AsyncAuthRequest().GetAwaiter().GetResult();
            }

            // only need the scheme and host, strip out the rest
            var serverUri = new Uri(_tokenResponse.InstanceUrl);
            var endpoint = $"{serverUri.Scheme}://{serverUri.Host}/cometd/45.0";

            var headers = new NameValueCollection { { nameof(HttpRequestHeader.Authorization), $"OAuth {_tokenResponse.AccessToken}" } };

            // Salesforce socket timeout during connection(CometD session) = 110 seconds
            var options = new Dictionary<string, object>(StringComparer.OrdinalIgnoreCase)
            {
                { ClientTransport.TIMEOUT_OPTION, _readTimeOutMs },
                { ClientTransport.MAX_NETWORK_DELAY_OPTION, _readTimeOutMs }
            };

            _clientTransport = new LongPollingTransport(options, headers);

            _bayeuxClient = new BayeuxClient(endpoint, _clientTransport);

            // adds logging and also raises an event to process reconnection to the server.
            _errorExtension = new ErrorExtension();
            _errorExtension.ConnectionError += ErrorExtension_ConnectionError;
            _errorExtension.ConnectionException += ErrorExtension_ConnectionException;
            _errorExtension.ConnectionMessage += ErrorExtension_ConnectionMessage;
            _bayeuxClient.AddExtension(_errorExtension);

            _replayIdExtension = new ReplayExtension();
            _bayeuxClient.AddExtension(_replayIdExtension);
            _bayeuxClient.Handshake();
            foreach (var listener in _listeners)
            {
                int realReplayId = 0;
                if(replayIdToUseOnReconnect == KnownInvalidReplayId)
                {
                    realReplayId = listener.ReplayId;
                }
                else
                {
                    realReplayId = replayIdToUseOnReconnect;
                    _logger.LogWarning($"ReplayId was previously invalid. Setting it to {realReplayId} for PushTopic: {listener.TopicName}");
                }
                SubscribeTopic(listener.TopicName, listener, realReplayId);
            }
            _logger.LogDebug("{name} was created...", nameof(BayeuxClient));
        }
    }

The logical cause would be hitting some of the "dead-end" else cases/ConnectionMessage/ConnectionException, but I can not locate those logs in application insights.

So I am asking here if anyone else have experienced this or have any pointers to what might have gone wrong.

*P.S I have since removed all the dead-end else cases and instead re-create the client in those cases; Since then I have not experienced one of the clients stopping.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.