GithubHelp home page GithubHelp logo

stidsborg / cleipnir.resilientfunctions Goto Github PK

View Code? Open in Web Editor NEW
36.0 2.0 4.0 3.29 MB

Implement resilient .NET code using ordinary functions & actions

License: MIT License

C# 100.00%
saga saga-pattern resilient-functions csharp dotnet resiliency micro-service fault-tolerant process-manager workflow-as-code

cleipnir.resilientfunctions's Introduction

.NET NuGet NuGet Changelog

logo
Simply making fault tolerant code simple

Cleipnir's Resilient Functions

Providing a simple way to ensure your code gets run - until you say it is done!

Resilient Functions is a simple and intuitive .NET framework for managing the execution of functions which must complete in their entirety despite: failures, restarts, deployments, versioning etc.

It automatically retries a function invocation until it completes potentially across process restarts and physical nodes.

The framework also supports postponing/suspending invocations or failing invocations for manually handling. Furthermore, versioning is natively supported.

It requires a minimal amount of setup to get started and seamlessly scales with multiple running instances.

Crucially, all this allows the saga pattern / process manager pattern to be implemented in a simple yet powerful way.

Out-of-the-box you also get:

  • synchronized invocation across multiple process instances
  • cloud independence & support for multiple databases
  • simple debuggability & testability
  • easy versioning of functions
  • native support for rpc and message-based communication
  • graceful-shutdown
What it is not?
Unlike other saga frameworks Resilient Functions does not require a message-broker to operate.
It is a fully self-contained solution - which operates on top of a database of choice or in-memory when testing.

Sections

Getting Started

Only three steps needs to be performed to get started.

Firstly, install the relevant nuget package (using either Postgres, SqlServer, MySQL or Azure Blob-storage as persistence layer). I.e.

dotnet add package Cleipnir.ResilientFunctions.PostgreSQL

Secondly, setup the framework:

 var store = new PostgreSqlFunctionStore(ConnStr);
 await store.Initialize();
 var functionsRegistry = new FunctionsRegistry(
   store,
   new Settings(
     unhandledExceptionHandler: e => Console.WriteLine($"Unhandled framework exception occured: '{e}'"),
     leaseLength: TimeSpan.FromSeconds(5)
   )
 );

Finally, register and invoke a function using the framework:

var actionRegistration = functionsRegistry.RegisterAction(
  functionTypeId: "OrderProcessor",
  async (Order order, Workflow workflow) => 
  {  
    await _paymentProviderClient.Reserve(order.CustomerId, state.TransactionId, order.TotalPrice);

    await workflow.Effect.Capture(
      "ShipProducts",
      work: () => _logisticsClient.ShipProducts(order.CustomerId, order.ProductIds),
      ResiliencyLevel.AtMostOnce
    );

    await _paymentProviderClient.Capture(state.TransactionId);
    await _emailClient.SendOrderConfirmation(order.CustomerId, order.ProductIds);
  }
);

var order = new Order(
  OrderId: "MK-4321",
  CustomerId: Guid.NewGuid(),
  ProductIds: new[] { Guid.NewGuid(), Guid.NewGuid() },
  TotalPrice: 123.5M
);

await actionRegistration.Invoke(order.OrderId, order);

Congrats, any non-completed Order flows are now automatically restarted by the framework.

Message-based solution:

It is also possible to implement message-based flows using the framework. I.e. awaiting 2 external messages before completing an invocation can be accomplished as follows:

 var rAction = functionsRegistry.RegisterAction(
  functionTypeId: "MessageWaitingFunc",
  async (string param, Workflow workflow) => 
  {
    var messages = await workflow.Messages;
    await messages
      .OfTypes<FundsReserved, InventoryLocked>()
      .Take(2)
      .Completion();
  }
);

Show me more code

In the following chapter several stand-alone examples are presented.

Hello-World

Firstly, the compulsory, ‘hello world’-example can be realized as follows:

var store = new InMemoryFunctionStore();
var functions = new FunctionsRegistry(store, unhandledExceptionHandler: Console.WriteLine);

var rFunc = functions.RegisterFunc(
  functionTypeId: "HelloWorld",
  inner: (string param) => param.ToUpper()
).Invoke;

var returned = await rFunc(functionInstanceId: "", param: "hello world");
Console.WriteLine($"Returned: '{returned}'");

Source Code

HTTP-call & database

Allright, not useful, here are a couple of simple, but common, use-cases.

Invoking a HTTP-endpoint and storing the response in a database table:

public static async Task RegisterAndInvoke(IDbConnection connection, IFunctionStore store)
{
  var functions = new FunctionsRegistry(store, new Settings(UnhandledExceptionHandler: Console.WriteLine));
  var httpClient = new HttpClient();

  var rAction = functions.RegisterAction(
    functionTypeId: "HttpAndDatabaseSaga",
    inner: async (Guid id) =>
    {
      var response = await httpClient.PostAsync(URL, new StringContent(id.ToString()));
      response.EnsureSuccessStatusCode();
      var content = await response.Content.ReadAsStringAsync();
      await connection.ExecuteAsync(
        "UPDATE Entity SET State=@State WHERE Id=@Id",
        new {State = content, Id = id}
      );
    }).Invoke;

  var id = Guid.NewGuid();
  await rAction(functionInstanceId: id.ToString(), param: id);
}

Source Code

Sending customer emails

Consider a travel agency which wants to send a promotional email to its customers:

public static class EmailSenderSaga
{
  public static async Task Start(MailAndRecipients mailAndRecipients, Workflow workflow)
  {
    var state = workflow.States.CreateOrGet<State>();  
    var (recipients, subject, content) = mailAndRecipients;

    using var client = new SmtpClient();
    await client.ConnectAsync("mail.smtpbucket.com", 8025);
        
    for (var atRecipient = state.AtRecipient; atRecipient < mailAndRecipients.Recipients.Count; atRecipient++)
    {
      var recipient = recipients[atRecipient];
      var message = new MimeMessage();
      message.To.Add(new MailboxAddress(recipient.Name, recipient.Address));
      message.From.Add(new MailboxAddress("The Travel Agency", "[email protected]"));

      message.Subject = subject;
      message.Body = new TextPart(TextFormat.Html) { Text = content };
      await client.SendAsync(message);

      state.AtRecipient = atRecipient;
      await state.Save();
    }
  }

  public class State : WorkflowState
  {
    public int AtRecipient { get; set; }
  }
}

Source Code

cleipnir.resilientfunctions's People

Contributors

stidsborg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

cleipnir.resilientfunctions's Issues

EventSource.Emit may Re-invoke RF

Overload EventSource,Emit allowing it to re-invoke an underlying Resilient Function which is either PostPoned or optionally Failed

Consider Retention Period Feature

Should it be possible to set up automatic deletion of sagas which have been successfully completed for some predetermined period of time?

Improve EventSource sematics

The event source should remember if it has thrown an exception will processering events and stay in a failed state after this point.
All invocations should throw the previous exception again.

Extend IFunctionStore with SetScrapbook-method

Implement method ala:
Task SetScrapbook(FunctionId functionId, string? scrapbookType, string scrapbookJson, int epoch);

This is in line with breaking the SetFunctionState-method up into smaller more specific methods

Clock Resolution Issue

Consider adding constant delay to postponed functions to compensate for poor clock resolution

FunctionId ToString

Consider changing FunctionId's ToString-method to be: "FunctionInstanceId@FunctionTypeId"

Allow re-entrance in RegisterRunningRFunc

It is sometimes beneficial to invoke RegisterRunningRFuncDisposable multiple times for the same running resilient function.
E.g. when starting to execute a postponed function or task from a watchdog.
At the moment the second invocation might throw an object disposed exception. An improvment would be for the second invocation to always succeed when there is a non-zero number of already running functions.

Scrapbook migration issue

Currently, a resilient function's scrapbook type is specified at creation and cannot be changed. It should be flexiable in the same way the parameter is.

Optimize Hot-path

Consider ways of optimizing hot-path. I.e. when no errors occur during the execution of a resilient function

Consider adding and using Unit type

There is some code duplication due to generic and non-generic implementations of classes.
Consider adding a Unit (void) type which can be used as a placeholder in the non-generic implemetation.
Thereby allowing non-generic implementations to use generic versions.

Drop the "R" Prefix.

Hi. Just wanted to add some feedback. I would drop the R prefix. RJob, R*. And RFunc rename to RFunction. You even have a class called RFuctions.

I would perhaps call RFunctions for FunctionContainer instead of Functions plural. Or even something else than functions plural :)

Introduce Scrapbook Versioning Issue

At the moment it is not possible to introduce a scrapbook parameter to an exsisting function through versioning.
Should this be allowed and if so how?

E.g.
V1: (string param) -> Task
V2: (string param, Scrapbook scrapbook) -> Task (all existing version 1 functions will fail)...

Implement Bulk Activate on FunctionStore

Add functionality allowing bulk of failed functions to be re-actived (status Executing) - thereby making it possible to easily bounce back after massive crash.

Bulk update something like:
Execute functions where where Type, Version and existing status is.

Consider TryToBecomeLeader Scrapbook json parameter

When updating a scrapbook consider making the update and becoming a leader one action.
That is by extending the TryToBecomeLeader-method:
Task TryToBecomeLeader(
FunctionId functionId,
Status newStatus,
int expectedEpoch,
int newEpoch,
long crashedCheckFrequency,
int version
);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.