cobaltfusion / debugvision Goto Github PK
View Code? Open in Web Editor NEWA Software Oscilloscope and portable set of data collection and visualization tools
License: Other
A Software Oscilloscope and portable set of data collection and visualization tools
License: Other
(emailed to me)
Thank you for a very interesting talk at Embo++.
I briefly mentioned that I had some input for your DebugVisualizer. I've tried to put it in writing below. However, after I've read the requirements you've put together at github, I realize that some it is already covered by your requirements. So, feel free to pick and drop as you like.
Anyway, I've been working on a couple of fairly large projects involving distributed control systems relying on wired (in one case) or wireless (another case) network technologies. The systems involve(d) between 20 and 30 communicating controllers. In each case, I've had access to a mixture of
In the one case, the log files for investigation came in from several hundred installations, and the challenge was to investigate whether the reported problems were already known or something new to be investigated further. In the other (current) case, the logs covered several days of testing from a single installation, and the issue was here to check that the latest bug fixes actually solved the identified problems and highlight new problems to be investigated. In such cases, it is essential that as much of the log file analysis is done automatically. In both projects, I've therefore created my own analysis scripts that traverse the logs to look for patterns that identify already known problems. This has worked reasonably well, although I probably still have some way to go before I've found the really good pattern for solving this kind of big-data/pattern analysis problems.
In the current project I work on, I've used Python. The analysis script invokes, for example, tshark to run various Lua script plugins inside tshark so that I can extract the information from Wireshark logs I'm interesting in. The script also uses matplotlib to automatically create plots of the some of the data from the identified problems. This eases manual verification of the analysis results, should it be needed.
The issue I'm having is, however, that my scripts are tailored to identify already known problems. Before reaching that state, I need to understand the problems. For each new problem discovered, there is a fair amount of manual analysis work. And this is where I'd ideally like a more visual tool to assist my work. I therefore found your graphical mock-up of the DebugVisualizer very interesting.
I've been thinking about adding something to the UI of sigrok.org (PulseView) as their UI seems to be offer some of what I was looking for, and I anyway want to be able to illustrate the events from my log files along with input/output to/from to controllers. I haven't done much to investigate this option, yet, however. But it looked promising.
I've also been thinking about setting up a JupyterLab notebook with access to my logs and some generic Python modules for analysis and printing of the log file contents. This would make it easy to share the workload among colleagues as the JupyterLab notebook can run on a central server and users only need a browser on their local machine. But this still seems a little tedious to work with as such a notebook is basically still just an interactive programming interface to Python (or Julia, R, C++/cling, etc.).
Some of my requested features of a debug visualization tool would be:
Here's another similar application:
https://github.com/zarunbal/LogExpert
I use it since years as more feature rich replacement of your already mentionend baretail
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.