swsnu / remon Goto Github PK

Java 30.52% Shell 0.37% CSS 2.36% JavaScript 60.72% Python 6.02%

remon's Introduction

ReMon (REEF Application Monitor)

4190.418A Web Enterprise Computing (Fall 2014) Term Project

Web client: http://remon-client.herokuapp.com/
Analytics: http://remon-client.herokuapp.com/analytics

Analyze REEF apps behaviour with a fresh tangy ReMon!

Have you built an application working on the distributed system? Have you spent all day to inspect logs from remote machines to analyze task status or debug errors? ReMon can relieve your pain and stress! You can see what is happening in your Application on one screen with enjoyable visualizations. Currently ReMon provides an interface compatible to REEF applications but it can be extended to support any kind of applications.

Structure

collector: ReMon Collector component
monitor: ReMon Monitor component
examples: Demo REEF applications
scripts: Useful scripts

Install

See our wiki. (https://github.com/swsnu/ReMon/wiki/)

remon's People

Contributors

Watchers

remon's Issues

Bug: the last metric value is not correct

Since we use async IO, we should check the time of each metric.

REST API provided by REEF webserver

This is description for the REST API provided by REEF webserver. We can use these for the base API of ReMon.

Uri Specification http://host:port/reef/v1
(All results are given as JSON format with REST API version 1)

/evaluators get all evaluator ids
/evaluators/?id={id}&... get the status of evaluator of id

/driver driverId, startTime, serviceInfoList

/close

/kill kill the driver
/logfile write out from the log file

Implement ReMon Collector to send the data

As a starting point, we are going to build Remon Collector to collect the evaluators’ status.
Two metric : Whether the evaluator is busy/available, Heap Memory Status (Used/Total)
We should elaborate what other metrics can be observed. The references will be posted in the Wiki

Reorganize Repository

As we changed our plan, we need to reorganize the repository.
The main changes are

Change the framework to tornado to make more scalable.
Use master branch instead of separating by the components.

This is quite urgent task to be done.

Document Update

As we made a lot of design changes yesterday, we need to update the document.
Reflect changes from the architecture and Framework.

Initialize Tornado webserver

This is part of #12

When #13 has been resolved, set up tornado webserver.

For testing, we may use heroku like we did in the last demo. Any thought?

Cache problem for static files

Changes are not reflected unless relaunching app. Fix me.

Apply new layout

Graphs + Message @ bottom

Implement Web UI

This can be divided sub-issues.

It contains some pages and components.

Client can choose an application to monitor.
At first, let's focus to show a graph with value from the database
Let's keep considering more useful UI components.

Control the appliation via Web UI

Remon will provide a way for user to control the application via web UI.

The first target control action is quit application.
Launching UDF is a future work to do.

Update documents for Iteration 3

We need progress report and so on.

Hotfix: show app name in title

Fix me

Hotfix: monitor test failed

Fix me

Use javascript template engine

Let's use handlebars! (http://handlebarsjs.com/)

Support fixed window graph

See http://code.shutterstock.com/rickshaw/examples/fixed.html

Support string message

Our logger class will look like this:

private val log = LoggerFactory.getLogger(this.getClass);
log.value(tag, value);
log.log(level, msg);
log.debug(msg);
log.info(msg);
log.error(msg);

Show current app status (running or finished)

Currently there is no way to know whether the app is running or not. We may need to change protocol to fix this.

Unit test for collector and monitor

I tried to write unit test code for collector and monitor.
I wrote simple unit test code for Collector and Codec.
(I'll push it after pull request #23.)
But it's hard to write unit test code for MetricSerializer and SimpleEchoSocket class. It looks that methods contained in these classes are related with data flush or websocket. I don't have idea what kind of unit test is need to these methods.
Please comment if you have any idea. Thank you.

Support time axis

We need to support time values.

Exception handling / security issues for monitor component

Currently our monitor component does not handle any unexpected inputs.

Support filter for the hierarchical graphs

There can be a lot of Metrics observed by ReMon. Some metrics can have a hierarchical structure among them. So, it would be useful to provide a way to specify the structure and visualize the data effectively.

Process multiple data points in a batch

Batch processing for speed up

Setup CI(Continuous Integration) testing

As CI testing tool, we are going to use Jenkins.

Set up Jenkins to build automatically for each time new pull request is created.

@TaeHunKim and I will work on this.

User stories

User Stories

0 Setup

Scenario: Connect REEF application to the ReMon Visualizer
- Given an REEF application code
- When I add some metrics to monitor
- And I connect to the ReMon Visualizer
- And I turn on the browser and connect to the Visualizer
- Then I can see the real-time analysis button on the top enabled
Scenario: Connect REEF application to the ReMon Aggregator
- Given an REEF application
- When I connect the application to the ReMon Aggregator
- And I connect the ReMon Aggregator to ReMon Visualizer
- Then I can see the some graphs are added with label ‘Aggregated’
- And I can see the Off-line analysis button on the top enabled.
- When I click the Off-line analysis button,
- Then I can see the log and profile panel on the screen.

1 Real-time analysis

Scenario: Monitor current snapshot of status of running application
- Given a REEF application and ReMon Visualizer running
- When I click the Real-time button on the main menu
- Then I can see the snapshot of current status of application in the ‘Snapshot’ panel
  - Possible metrics : Status - Running / Waiting / Finished …
  - How many evaluators the app has
  - Memory usage / CPU Usage for each evaluator
Scenario: Monitor time-series data of running application with dynamic refreshing view
- Given a REEF application and ReMon Visualizer running
- When I click the Real-time button on the main menu
- Then I can see the dynamically changing graph on the ‘Time-series’ panel
Scenario: Change time scale on the graph
- Given I am on the ReMon Page
- When I drag along the x-axes of the graphs
- Then I can see the change on the graph with different time-scale and data
Scenario: Restart REEF Application
- Given I am on the ReMon ‘Real-time’ panel
- When I click the ‘Restart’ button
- Then I can see the application restarts
Scenario: Quit running Task of one evaluator
- Given I am on the ReMon ‘Real-time’ panel
- When I click the ‘Quit’ button
- Then I can see the application restarts
Scenario: Register User Defined Functions(UDF) to ReMon
- Given I registered a function ‘foo’ via ReMon library
- When I am on the ReMon ‘Real-time’ panel
- Then I can see the button ‘foo’
Scenario: Execute User Defined Functions(UDF) via ReMon Web UI
- Given I registered a function ‘foo’ via ReMon library
- When I am on the ReMon ‘Real-time’ panel
- And I click the ‘foo’ button
- Then I can see the application does the things I expected to

2 Off-line analysis

Scenario: Investigate the log data
- Given an REEF application
- When I connect the application to the ReMon Aggregator
- And I connect the ReMon Aggregator to ReMon Visualizer
- Then I can see the some graphs are added with label ‘Aggregated’
- And I can see the Off-line analysis button on the top enabled.
- When I click the Off-line analysis button,
- Then I can see the log and profile panel on the screen.
Scenario: Profile with some metrics via log data
- Given an REEF application
- When I have ReMon Aggregator working
- And I am on the page of Off-line analysis
- Then I can see the graph generated with the metrics I registered in the ReMon library

Implement ReMon Monitor

ReMon monitor is to Store the data into Database and show the data to the client. I will register sub-issues related to this component.

Feature: automatic websocket reconnection

Sometimes websocket connection is suddenly closed. I want to add reconnect button.

Add ML application for demo

Machine Learning application is good to show our awesomeness.

Analytics viewer

Extension of issue #39. We need a viewer for analytics.

Make a scripts directory

To run REEF application
To run test codes
Dummy client for debugging

Bug: message box is not cleared when returning to app list page

Fix me 😄

Asynchronous Database IO

Change pymongo to motor which supports tornado's coroutine.
See http://motor.readthedocs.org/en/stable/examples/callbacks-and-coroutines.html

Edit Jenkins test script

Currently build(test) script in Jenkins is written as below:

#Test for Collector
cd "$JENKINS_HOME/jobs/ReMon Build/workspace/collector"
mvn test

#Test for Monitor
cd "$JENKINS_HOME/jobs/ReMon Build/workspace/monitor"
python -m tornado.testing tests.runtests

Test script of Monitor can be useless when file name of test code is changed. We can change the script to test correctly even if the file name of test code is changed.

Histogram visualization

Use cases for Histogram / CDF visualization.

Make separate Configuration for Collector

In the Collector part, the objects are created via injection but there is no configuration to specify the behaviour. I think we need to define and specify the configuration for Collector classes.

Hotfix: show the websocket connection status

Sometimes the connection suddenly closed. It would be better to show the current status, and reconnect if possible.

Authentication for clear method

Use auth_key

Support multiple lifecycle event for same tag

Show page loader when connecting websocket

Integration Test

After #12 and #11 are resolved, then we should perform integration test to make sure our system works well.

More fine-grained test scenarios and plan should be established. Let's improve our test plan and update the plan to the design doc.

Task completion time 측정

데모할 때 어떤 유즈케이스에서 사용할 수 있을지 보여주기

Database design & setup

This is part of #12

We are going to use MongoDB.

After #13 and #14 is done, set up MongoDB and connect it to Tornado.

Motor(mongoDB+tornado) seems popular as I searched, but there might be a better way.

Bug: the message log is not sorted

Since we use async IO, we should check the time of each message.

Fix data base unit from list to element

Support multiple application in User Interface

As described in #17, Client should be able to choose application to watch among the multiple running REEF applications. Followings are the instructions I think:

When access to the Remon main web UI, then user see the identifier of applications(probably one of id / name / address?)
Next to the application name user can see the status for it : running / finished.
When user clicks the app identifier then move to the page specified for the application.

UI may differ from how I imagined, but the things above seem to be essential parts.

Monitor user behavior using an Analytics framework.

This is part of Programming assignment 2.

We can choose between two options described on https://sites.google.com/site/snuwecfall2014/warm-up-programming-assignment-2 ; We can just use the existing framework or we may build one on our own.

@wynot12 and @elnn will work on this.

So it would have dependency to the visualization module built in JS.