GithubHelp home page GithubHelp logo

alexxnica / noflo-runtime-msgflo Goto Github PK

View Code? Open in Web Editor NEW

This project forked from noflo/noflo-runtime-msgflo

0.0 1.0 0.0 154 KB

NoFlo runtime for AMQP/MQTT

JavaScript 0.44% CoffeeScript 99.56%

noflo-runtime-msgflo's Introduction

noflo-runtime-msgflo

NoFlo runtime, designed for use with msgflo. Loads a NoFlo graph and registers it as a msgflo participant. The exported ports of the NoFlo network then becomes accessible through message queues (AMQP or MQTT), and msgflo can coordinate how multiple such networks send data between eachother.

Will eventually allow a single entry-point for FBP runtime protocol clients (like Flowhub) to access NoFlo FBP networks that span multiple nodes.

Usage

Using the commandline tool

noflo-runtime-msgflo --name myworker --graph project/WorkerGraph --broker amqp://foo.cloudamqp.com/bar

Altenatively one can use the embedding API, see src/mount.coffee

runtime = require 'noflo-runtime-msgflo'
options =
  name: 'gssmeasure'
  graph: 'the-grid-api/PrecomputeGss',
  basedir: __dirname
  broker: config.amqp.url,
rt = new runtime.mount.Mounter options
rt.start (err, rt) ->
  # started

Debugging

noflo-runtime-msgflo supports flowtrace allows to trace & store the execution of the NoFlo network, so you can debug any issues that would occur. You can enable tracing using --trace commandline argument, or via the trace:start FBP runtime protocol message.

noflo-runtime-msgflo --graph project/MyMainGraph --trace

To trigger dumping a flowtrace locally, send the SIGUSR2 Unix signal

kill -SIGUSR2 $PID_OF_PROCESS
... Wrote flowtrace to: /tmp/1151020-12063-ami5vq.json

Or, to trigger a flowtrace remotely, send a trace:dump FBP protocol message to the queue of participant.

msgflo-send-message --queue .fbp.$participantId.receive --json '{ "protocol":"trace", "command":"dump", "payload":{ "graph":"default", "type":"flowtrace.json"} }'

Then, assuming .fbp.$participantId.send has been bound to queue my-flowtraces, one could download it

msgflo-dump-message --queue my-flowtraces --amount 1

You can now use various flowtrace tools to introspect the data. For instance, you can get a human readable log using flowtrace-show

flowtrace-show /tmp/1151020-12063-ami5vq.json

-> IN repeat CONN
-> IN repeat DATA hello world
-> IN stdout CONN
-> IN stdout DATA hello world
-> IN repeat DISC
-> IN stdout DISC

Optimizing startup time

Since NoFlo 0.7 FBP manifest is used, making it possible to cache components which turns process startup faster due to less disk IO. To cache graph's components, first install noflo and run noflo-cache-preheat to create the FBP manifest file fbp.json. Then run the graph passing the cache parameter:

noflo-runtime-msgflo --cache true --name myworker --graph project/WorkerGraph --broker amqp://foo.cloudamqp.com/bar

TODO

0.3

noflo-runtime-msgflo's People

Contributors

jonnor avatar bergie avatar greenkeeperio-bot avatar greenkeeper[bot] avatar automata avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.