iqtlabs / crviz Goto Github PK
View Code? Open in Web Editor NEWBrowser-based visualization tool that uses JSON and an interactive enclosure diagram to visualize networks.
Home Page: https://iqtlabs.github.io/CRviz/
License: Apache License 2.0
Browser-based visualization tool that uses JSON and an interactive enclosure diagram to visualize networks.
Home Page: https://iqtlabs.github.io/CRviz/
License: Apache License 2.0
There's an issue importing CSV files when a quoted property in a CSV data object contains EOL characters. i.e.,
ID,name,description
123,Something,"This line\nhas a linebreak."
The linebreak will cause the CSV import to fail.
It would be nice if the URL add breadcrumbs when elements are changed in the control panel, so that someone could use the URL to get all of their particular settings back without having to click through all of them in the interface every time.
Indexing a dataset is inherently a relatively intensive process. There is no reason this can't be done in the background while the user is otherwise able to use and interact with the application. This operation should be implemented as an RxJS epic so that if a new search library is introduced then there is a single point to insert appropriate index creation logic and if a search index is not used there is no need to dispatch the action.
Add checks to enzyme tests to ensure that components receive correct css classes
When automated PRs are created (by things like Greenkeeper and Snyk) it would be nice to have tests that fail due to meaningful changes in appearance.
implement a data strategy (probably a redux epic) to handle filtering datasets based on certain user defined criteria.
Upload/Fetch->Load->Format->Index->Set Hierarchy->Display
Upload/Fetch->Load->Format->Index->Filter->Set Hierarchy->Display
It would be great if we could search on the data once it's loaded. Minimally I was thinking:
currently our dev and docker process involves a mix of npm and yarn. They both do the same general things and it is wasteful and confusing to use both. I think we need to pick one and go with it. My intention is to do some research before making a specific suggestion. Do any team members have a strong opinion one way or the other?
currently, if there are updates to the file that is being fed by a URL selection, you have to refresh the whole page and load the URL in again to get the updated dataset, if there was a polling feature for this that updated the interface with any data updates on some interval, it would be useful for dashboard/HUD setups.
Allow a user to add authorization credentials when selecting Custom URL as a datasource
perhaps by entering:
After the following sequence:
At this point, the canvas area will appear to be blank. What is actually happening is that there are only nodes to display, but CRViz has been instructed not to show those. This will be confusing to some users, we should probably reset the value of Show nodes to true on these events in order to avoid confusion
this would be particularly useful when using a URL where you may not already have the JSON file of what you're viewing
Calculate and display statistics on the properties of the dataset being displayed by CRviz, such as
Allow user to upload a CSV file; convert it to json format and feed it to CRviz.
Modify the data ingest to accept multiple datsets representing multiple points in time.
I think the smartest thing to do here is to set the backend/data side up to accept an array of arbitrary length and then at the UI level restrict the components to looking at 2 points in the dataset. Probably just the first 2 at least to start, but there's no reason we couldn't give the users a way to say i want to see frame 1 compared to frame 17. Plus that leaves us in the position of not needing to upgrade the backend if we come up with a clever way to display things over time
Additionally the Epics will need to take in some sort of key designation for state storage so that we have an authoritative way to determine what objects at timestamp A correspond to at timestampB.
When using the refresh functionality there is no indicator to the end user that it's doing anything.
Clicking refresh button should ideally either have a spinner or some sort of 'working on it' or 'triggered' notification to the end-user so that they know it's doing something.
Without this, it's not obvious whether or not refresh actually did anything until it happens (assuming the refresh has new data).
Environment: https://cyberreboot.github.io/CRviz/
Steps to Recreate:
Expected Behavior: CRViz should switch to using the new dataset resetting the Color By field
Actual Results: the screen goes blank and an error can be seen in the javascript console
TypeError: e.legend is undefined
Add unit tests covering some edge and error cases
Instead of clicking a node to zoom to it use a standard set of zoom in/out buttons. This would give finer and more intuitive control over the level of zoom and free the mouse click gesture up to be used for some other feature.
Allow users to add a layer of markup on top of the displayed visualization.
When presenting data visualized by CRviz to other audiences it would be exceptionally useful to have some ability to highlight or otherwise direct attention to some particular item or group of nodes.
Additionally this feature could allow users to display supplemental or contextual information. Ideally there should be a mechanism to export the markup as an image, possibly without displaying the controls.
If you pass a single object instead of a dataset to CRViz instead of labelling the fields with values it will return indices.
tep 6/11 : RUN yarn install ---> Running in 7368cd02a803 yarn install v1.7.0 [1/4] Resolving packages... (node:6) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead. warning [email protected]: breaking changes in minor version [2/4] Fetching packages... info [email protected]: The platform "linux" is incompatible with this module. info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation. warning [email protected]: The engine "gitbook" appears to be invalid. [3/4] Linking dependencies... warning " > @fortawesome/[email protected]" has unmet peer dependency "prop-types@^15.5.10". warning "react-scripts > babel-preset-react-app > babel-plugin-transform-dynamic-import > @babel/[email protected]" has incorrect peer dependency "@babel/[email protected]". [4/4] Building fresh packages... success Saved lockfile. Done in 29.97s.
AND
Step 8/11 : RUN npm install -g serve ---> Running in f93ccc900906 npm WARN notice [SECURITY] lodash has the following vulnerability: 1 low. Go here for more details: https://nodesecurity.io/advisories?search=lodash&version=2.4.2 - Run
npm i npm@latest -gto upgrade your npm version, and then
npm audit` to get more info.
/usr/local/bin/serve -> /usr/local/lib/node_modules/serve/bin/serve.js
hostReportError.js:8 Uncaught TypeError: Cannot read property 'map' of undefined
at A.Builder.<anonymous> (index-dataset-epic.js:80)
at t (lunr.js:53)
at e.v [as project] (index-dataset-epic.js:78)
at e._next (map.js:75)
at e.next (Subscriber.js:93)
at t._subscribe (scalar.js:5)
at t.subscribe (Observable.js:161)
at t.call (map.js:51)
at t.subscribe (Observable.js:158)
at t.call (map.js:51)
latest in master @c01c387eb1600023e7e619b2fe34ccd3e790a643
chrome 65.0.3325.181
Run Poseidon and go to the instance of CRviz and select `Poseidon Network`, you'll see the above error in the Developer Console of Chrome
What did you think would happen?
Not throw an error
What really happened?
hostReportError.js:8 Uncaught TypeError: Cannot read property 'map' of undefined
at A.Builder.<anonymous> (index-dataset-epic.js:80)
at t (lunr.js:53)
at e.v [as project] (index-dataset-epic.js:78)
at e._next (map.js:75)
at e.next (Subscriber.js:93)
at t._subscribe (scalar.js:5)
at t.subscribe (Observable.js:161)
at t.call (map.js:51)
at t.subscribe (Observable.js:158)
at t.call (map.js:51)
We really should display version info somewhere in the application. We have it up on github.io and we want outside users to report issues against it then we really should provide a mechanism for everyone to know what version of the software is being used.
clear search results in an error
Create UI components to allow users to create, apply, and remove filters
because of an id conflict in the upload control, the htmlFor would always find the first instance of a fileUpload element in the DOM tree, leading to confusing behavior from this
in the upload event
the filename would populate in the upload area of the second dataset control.
the filename would always populate in the upload area of the first dataset control.
Modify the dataset select and upload UI to accept data in accordance with the specifications of #108 and to accept entry of a keying field for use in assigning object identity. It may be necessary to move some of the components into a modal dialog of some sort, or isolate certain UI elements into collapsible panels.
if you search for say foo:bar
and foo
isn't a valid field, it will break any subsequent search queries
Implement a way to display time series change with the context of an enclosure diagram.
The visualization should:
refreshing the dataset resets all group by and color by settings.
the updated data should be displayed with the selected settings
the updated data is displayed as if on initial load
we should implement UI level test scripts. We just had a bug that @lilchurro found when reviewing PR #73 where updating dependencies changed visual behavior. It was minor and easily fixed but we need to prevent this kind of thing in the future. It looks like the default for react is a combination of Jest and Enzyme but I'll dig into it more before I implement anything. It looks like storybook is another popular option.
Describe your issue here.
the same behavior as:
the grouping nodes turn black (or white if using the dark theme)
Allow users to customize the colors when coloring by a specified attribute.
When presenting data visualized in CRviz the arbitrarily assigned colors can inadvertently undercut the message you are trying to convey. For instance suppose a dataset has a property "Risk" with values low, medium, high, and critical. When coloring by this attribute it is possible for the colors to be assigned as
Risk | Color |
---|---|
Low | Red |
Medium | Orange |
High | Green |
Critical | Blue |
which is going to undercut the urgency generally associated with something considered a critical risk.
This would nicely complement the feature outlined in #103
Add google analytics to gh-pages branch
RxJS 6 has discarded operator chaining in favor of operator piping
(for good reason see: https://github.com/ReactiveX/rxjs/blob/91088dae1df097be2370c73300ffa11b27fd0100/doc/pipeable-operators.md for more detail)
needless to say this presents several breaking changes for CRViz in that it breaks ALL of our epics. In the interests of smaller more performant js bundles and objects and eliminating unecessary dependencies we should upgade to RxJS 6.
Describe your issue here.
that the previously selected dataset information would be populated
the dataset panel looked the same as it does on initial load.
Errors are frequently reported using window.alert("message"). This is bad practice and difficult to test.
N/A
Errors should be collected and displayed in a fashion that is:
window.alert was used
Found in version: 0.2.1
Steps to reproduce:
run the command: docker build -t crviz-gh -f Dockerfile.gh .
Expected result
the docker image builds successfully
Actual result:
received the error: /bin/sh: git: not found
The command '/bin/sh -c git config --global user.email " "' returned a non-zero code: 127
Write a quick start/FAQ type of guide for new users outlining the various features of CRviz and how to use them. Ideally it should be hosted on the web somewhere so that we can link to it both externally and from within the application. Maybe a markdown file in github so that we can version it alongside the application?
when searching using the notation "fieldaName: value" the capitalization of the fieldName must match exactly or the search will fail.
Add docker build to travis config so that breaking changes to the docker build process are visible.
Remove the default to red for unknown nodes
Red is typically used to denote a condition that is bad or requires remediation, unknown node does not necessarily fit this bill
As implemented, this is a hidden, undocumented behavior that only fits certain types of data within certain types of datasets. It is also the only scenario where anything is color coded by default, which is a sharp break with the way the rest of the application is designed.
N/A
Steps to recreate:
The user will receive the following error in the javascript console:
TypeError: e.legend is undefined
Implement a mechanism to allow users to fix the tooltip even after the mouse moves off of the tooltip's target node to allow better analysis and the ability to copy data from the tooltip.
Currently CRviz requires that within a JSON file upload the JSON is prefaced with:
{ "dataset": [
I believe this is required for some of the additional filtering/import options when bringing in data from an HTTP-based source. That said, it would be great if we could introduce some logic that could take a file with the appropriate "dataset" prefixing, but in the absence of that, just assume that the file is stock JSON and attempt to import it as such. (i.e. sans "dataset" reference)
This is specifically with a local file upload in CRViz - we probably want to keep the URL-based import as-is.
As it stands, many of our epics catch errors and display them to the user using window.alert
this
It would be better to use this logic to trap and transform any errors that we need to and then allow the results to bubble up to our components which should handle the error at the UI level.
Give users an ability to export the data and/or their view of the data
In order to create shareability, but to also avoid the headaches of having to create a backend architecture to support shareable URLs, we should allow control state to be loaded from the uploaded data file. Additionally, this export must implicitly contain the data under view because the existing architecture fetches the data from a url so the only way to ensure that the user is sharing the exact data that the user is viewing is to package into one file.
This would also satisfy #50. It may be possible/make sense to:
None
A button or function to reset the display clearing anything like coloring, hierarchy selection, zoom, etc. with the exception of the loaded dataset since the user might be pulling from a remote url and will not want the underlying data reset.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.