GithubHelp home page GithubHelp logo

salboaie / swarmesb Goto Github PK

View Code? Open in Web Editor NEW
146.0 26.0 34.0 8.22 MB

A light, open source, ESB or message hub for node.js. Using the "swarm communication pattern"

JavaScript 36.07% HTML 30.07% Shell 0.04% CSS 33.81%

swarmesb's Introduction

#SwarmESB central repository. Refactoring and master plan: This is the new version of SwarmESB, based on [SwarmCore] (https://github.com/salboaie/swarmcore "SwarmCore") that adds:

  • default load distribution between Adapters (done)
  • new broadcast primitive (done)
  • "do" syntax to describe guaranteed message delivery (at least once)
  • allow creation of personalised/custom ESBs that can use other middleware than Redis (done)
  • refactoring in progress to grow it around a DI container for better stability (almoust done)
  • based on a new powerful ACl module (acl-magic module/done)
  • deployment support for docker containers (done)
  • UI with system monitoring (done)
  • better logging (done)
  • support for typed persistence, complex custom type representation, privacy ontologies ("apersistence" module)
  • support for executable choreography (done, more improvements in progress)
  • distributed transactions mechanisms (planned)
  • support for representation of long living processes (planned)

What is SwarmESB ?

Swarm communication is a micro-services architecture based on a new message communication pattern we call "swarms" It offers an starting architecture for cloud and enterprise systems. The usual method of doing message passing is to have relatively intelligent processes, objects or actors sending and receiving dumb messages. Thinking to messages as relatively smart beings visiting relatively non intelligent places, we noticed a positive effect on the quality of the code and a new path for decomposing complex applications in small services. SwarmESB is using swarm communication for creating an easy to grow Enterprise Service Bus. You add your own microservices (we call adapters) and describe how microservices get composed for your usecases in a coupling free DSL (swarm descriptions). SwarmESB it is groing in the direction of executable coreographies to offer by design an improved architecture for security and privacy .

Presentation: https://docs.google.com/presentation/d/1xn1L9bbiD1TUY7ku5d0V_zF7SBkH3ANQ2uN8AxAXaA8/

Overview

A "swarm" is a set of related messages with some basic intelligence and is based on an intuitive point of view: computer processes communicating by asynchronous messages are more like "dumb trees/flowers" visited by "smart swarms of bees" than "smart people" communicating by "dumb messages". This intuition have a powerful effect on reducing complexity of many distributed systems (already proved by SOA) Service orchestration with SwarmESB is elegant, there is no need to learn new languages and difficult concepts: just use Java Script code and a few simple concepts.

SwarmESB is an open source project implementing a new approach: "Swarming architecture" that you could reuse for your new multi-tenant systems that will transparently acquire the benefits of scalability, high availability, highly parallel computing and loose coupling usually obtained with asynchronous messages, message queues, pub/sub channels and using a message or service bus. While this project is build using Node.js and Redis, this project will present "swarming" as a new concept, comparable, but simpler than direct usage of Enterprise Integration Patterns or the Actor model used in Scala and Erlang.

Examples

A swarm description is written in Java Script and is composed from: variable declarations (for defaults), constructors (functions that get called on the adapter that starts a swarm) and phases (code that get executed remotely, usually in another node)

The swarm described bellow will magically get executed without any other programming efforts in 3 nodes (processes):

        vars:{
            message:"Hello World",
        },
        start:function(){ //constructor  that can be executed in any adapter
                    this.swarm("concat");  // swarm is a "primitive" used to invoke execution in a phase
                },
        concat:{ // phase that get executed in "Core" adapter
            node:"Core",
            code : function (){
                    this.message = this.message + " The swarming has begun! ";
                    this.swarm("print");    //move again
                }
        },
        print:{ //print phase executed in "Logger" adapter
        node:"Logger",
        code : function (){
            cprint(this.message);    //use of some api, specific to the Logger node
            },
        }

See: each node could be on a different continent or hosted by a different organisation!

Easy and fun!

Check our wiki: https://github.com/salboaie/SwarmESB/wiki

Technologies: node.js, Redis, PUB/SUB

SwarmESB ca be used as a light, open source, ESB (or Message Bus) for your enterprise applications. Instead of message oriented communication or web services, you describe your communication between "nodes" in what we call "swarm descriptions" or simple "swarms". "Nodes" can be adapters to various servers or clients connected to the "swarming middleware": SwarmESB.

Your adapters can use web services as a particular case of providing existing APIs in a node.

Conclusions:

1. Maintainable code: Open/closed principle, no threads get abused (ever), simple
2. High performance: We are using node.js and his asynchronous capabilities revealed no throughput degradation in
our benchmarks. Network latency have no effect in capability of the system to execute things in parallel.
3. Scalability: Load balancing and sharding (research) can be implemented using swarming concepts
4. High availability: Still at the research level but looks promising

#Copyright (c) Axiologic Research & Alboaie Sînică. Code License: LGPL or MIT The content of the README and of the wiki is Public Domain

Commercial versions derived from this project are allowed. However, it would be desirable that all ESBs developed around swarm communication idea to get identical APIs and compatible programming styles. Contact [email protected] for details.

#Supporters WebStorm, the best IDE for node.js and JS

A free book about swarms

swarmesb's People

Contributors

asaccool avatar catalinmanolescu avatar cprnt avatar jwulf avatar mastaleru avatar salboaie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

swarmesb's Issues

Can you make the install a little easier?

Maybe an example? For example, I'm a newbie but have installed dozens of node apps and set up dev environments without a hitch, but just can't get your ESB installed and sadly need to continue my search for a good answer to a multiple-endpoint transaction processing service I need to set up (if your looking for a solid POC for your ESB I may have what you're looking for btw).

Install instructions are over my head, for example when you talk about setting path to etc is that my system etc on mac or the etc or in the package?...if system this is the first node package outside of nvm I've seen that requires this type of config.

Anyway, if this package is intended for degreed engineers then I apologize and cary on. If you would like to get it in more hands to check out I suggest making the install instructions a 1-2-3 done with actual examples on config file requirements...

WebClientAdapter.js missing?

I followed the installation instructions here: https://github.com/salboaie/SwarmESB/wiki/Install-guide

When I run [~/workspace/SwarmESB] $ node adapters/Launcher.js, I get:

[24-5,0:48][Launcher] WebClientAdapter need restart[6].Not alive.
module.js:338
throw err;
^
Error: Cannot find module '/Users/sitapati/workspace/SwarmESB/adapters/WebClientAdapter.js'
at Function.Module._resolveFilename (module.js:336:15)
at Function.Module._load (module.js:278:25)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3

Launcher will not die

I cannot kill the launcher. When I hit ^C I get the following:

^C[24-5,0:54][Launcher] Got SIGINT.
Retrying swarm propagation towards SwarmCoreExample:Logger: {"meta":{"name":"log.js","swarmingName":"log.js","currentPhase":"doLog","command":"phase","swarmCounter":"Launcher:263"},"message":"Got SIGINT.","level":"ERROR","userId":null,"debug":"false","nodeName":"Launcher","stack":""}
Retrying swarm propagation towards SwarmCoreExample:Logger: {"meta":{"name":"log.js","swarmingName":"log.js","currentPhase":"doLog","command":"phase","swarmCounter":"Launcher:264","timout":1000},"message":"Got SIGINT.","level":"ERROR","userId":null,"debug":"false","nodeName":"Launcher","stack":""}

APP require 'swarmcore' module

When trying to launch, getting the following message:
Cannot find module 'swarmcore'

Error: Cannot find module 'swarmcore'
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:278:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/www/SwarmESB/adapters/demoLaunch.js:5:12)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)

The APP does not start.

Installation of swarmcore through does not help either.

Adapter packaging format

I have written a Google Calendar microservice Adapter that I would like to contribute.

I will take the calendar URL and authentication token and isolate it from the service code via injection.

I think it would make sense to have a pluggable adapter architecture - so ultimately you can do something like:

swarmesb install google-calendar

and have a Google Calendar adapter show up in your adapters folder, automatically added to your Launcher.js.

I propose this as a starting point:

adapters
      |-----GoogleCalendarAdapter.js   
      |------.packages
                   |---google-calendar
                            |---README.md
                            |---swarm-package-meta.json
                            |-------code
                                       |-----node_modules
                                       |-----GoogleCalendarAdapterImplementation.js

config.js is where the end-user (developer) configurable pieces are, like authentication credentials, calendar urls, etc..

I'll have to experiment a bit to see how injection and multiple calendars and all that kind of stuff work. I think that GoogleCalendarAdapter.js could have stubs in it that call functions in the underlying GoogleCalendarAdapterImplementation.js library, and developers can write their custom code in GoogleCalendarAdapter.js. It would make sense for that custom code to be outside the auto-magically installed package, to keep that "isolation between infrastructure and innovation".

This repo should be deprecated

At the moment, absolutely nothing works. So if the active development has ended, please mark this repo read only or deprecated, if the required dependencies or the actual app is no longer usable.

Error loading index.html

Hi,
I'm trying to evaluate SwarmESB - On a Mac. (Supported?).

When I browse to localhost:8080, I get "Error loading index.html" (I get 8080 from the console output, not 8081 as in the install docs). The console also reports the following error.

Error: Cannot find module 'swarm-winston' at Function.Module._resolveFilename (module.js:469:15) at Function.Module._load (module.js:417:25) at Module.require (module.js:497:17) at require (internal/module.js:20:19) at Object.<anonymous> (/Users/smgary/Downloads/SwarmESB/node_modules/swarmcore/etc/adapters/DefaultLogger.js:8:15) ....

MySQL dependency

The installation docs should be updated since it seems that MySQL is a dependency.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.