sirixdb / sirix-web-frontend

A web front-end for SirixDB based on Nuxt.js/Vue.js, D3.js and Typescript

Join us on Slack | Community Forum

Working on your first Pull Request? You can learn how from this free series How to Contribute to an Open Source Project on GitHub and another tutorial: How YOU can contribute to OSS, a beginners guide

SirixDB Web frontend - An Evolutionary, Versioned, Temporal NoSQL Document Store

Store and query revisions of your data efficiently

"Remember that you're lucky, even if you don't think you are, because there's always something that you can be thankful for." - Esther Grace Earl (http://tswgo.org)

Introduction

Discuss it in the Community Forum

This is the repository for a web frontend based on Vue.js, D3.js and TypeScript.

It'll provide several interaction possibilities to store, update and query databases in SirixDB. Furthermore the front-end will provide interactive visualizations for exploring and comparing revisions of resources stored in SirixDB based on different views.

Some ideas for comparing revisions of the XML or JSON resources, which need to be ported from a Java GUI/Processing can be found in my Master's Theses and in a Screencast.

Test and edit in Gitpod

Test and edit this project in a web based VS code environment.

Project setup

We currently face the following issue, whereas this workaround works until Nuxt.js or Element-UI fix the issue

We've created a Dockerfile and a docker-compose.yml file to simplify the setup process. However you still have to setup Keycloak as desribed in the documentation for the REST-API (especially assigning the roles to an admin-user):

In order to use docker-compose:

1. git clone https://github.com/sirixdb/sirix-web-frontend.git
2. Start Keyclock with the command sudo docker-compose up waitforkeycloak from your sirixdb directory with the copied docker-compose.yml file.
3. In your browser navigate to http://127.0.0.1:8080 and click on the link "Administration Console" (if using Docker Toolbox, see below).
4. Use username "admin", password "admin" to log in.
5. Navigate to Clients => sirix.
6. Set Standard Flow Enabled.
7. Set redirect URL to http://localhost:3005/callback.
8. Navigate to the Credentials tab and generate a new secret.
9. Put this secret in sirix/bundles/sirix-rest-api/src/main/resources/sirix-conf.json as the value for client.secret. Make sure that the oAuthFlowType is set to AUTH_CODE
10. Create a user: Users => Add User => Username admin. Under Credentials => New password => admin / admin. Set Temporary to off. Under Role Mappings add the 4 roles: create, delete, modify, view.
11. Start the SirixDB HTTP-Server with the command sudo docker-compose up -d server.
12. Start the Node.js Server without Docker: npm run dev.
13. In your browser call http://localhost:3005 and the frontend should appear.

NOTE: For those using Docker Toolbox instead of Docker to run Keycloak and Sirix, you will need to do the following:

1. Find your Docker IP (you should see it when starting Docker Quickstart Terminal).
2. Go to the nuxt.config.js file, and change

proxy: {
    '/sirix': {
      target: 'https://localhost:9443',
      pathRewrite: {'^/sirix': ''},
      agent: new Agent({ rejectUnauthorized: false })
    }
  }

to

  proxy: {
    '/sirix': {
      target: 'https://<your docker host>:9443',
      pathRewrite: {'^/sirix': ''},
      agent: new Agent({ rejectUnauthorized: false })
    }
  }

3. The URL for Keycloak in step 3 above is http://<your docker host>:8080.
4. You may also need to use a browser extension like switcheroo to redirect localhost:8080/ to <your docker host>:8080/.

Setting up the Nuxt.js application:

# install dependencies
$ npm install

# serve with hot reload at localhost:3005
$ npm run dev

# build for production and launch server
$ npm run build
$ npm run start

# generate static project
$ npm run generate

Customize configuration

For detailed explanation on how things work, check out Nuxt.js docs.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Simon Holdorf 💻

yang 💻

Atman 💻

Jawahar 💻

Moshe Uminer 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

sirix-web-frontend's Issues

TableView / Loading data on demand

As SirixDB is able to store Integer.MAX - 1 revisions, we have to take paging into account.

So the table should be loaded at first with the metadata from most-recent-revision-number to most-recent-revision-number - N revisions, to display the metadata from the most recent n revisions. Then on page 2 or when scrolling down the next 10 should be requested from the HTTP-Server...

Basic "beautiful" layout

We need a beautiful layout for the whole web app.

The basic functionality is just about different views, which either display a revision or comparisons between revisions.

TreemapView with D3.js

In order to get an overview about the stored XML and JSON resources we might want to use space filling approaches for tree-structured data as for instancdd a TreeMap.

TreeView to display databases and resources

Displaying databases and resources in a TreeView similarly to other database system front-end as for instance pgadmin. We can use the password credentials flow in the first place with an admin/admin username/pass posted to the login endpoint. We need to change this obviously in the next step.

1. Setup Keycloak and the SirixDB HTTP-server
2. Get a token by POST https://localhost:9433/login with JSON data:

{
  "username": "admin",
  "password": "admin"
}

3. GET https://localhost:9443: provide HTTP authorization bearer token, which you got in step 2.

To get a simple list of databases {"databases": [{"db1": "xml", "db2": "json"]}

4. If a user expands and logs in to a database (tree-view is clicked): provide HTTP authorization bearer token, which you got in step 2.

5. Then we can link the JSON or XML tree-view... and the other views in the future.

The type definition of `$loading` in Nuxt.js conflicts with element-ui

See: nuxt/typescript#49

I did the workaround locally mentioned in

nuxt/typescript#49 (comment)

However, I hope that gets fixed. Either from the Element-UI or Next.js developers...

Rewrite JavaScript code to TypeScript

In the long run I think it might be better to only have TypeScript based files.

Query view / query component

We need a way to simply query the backend and show the serialized form of the results.

SunburstView with D3.js

We want to be able to display huge JSON and XML instances (provide an overview), that is one revision.

Then it's all about visualizing the changes between revisions: In a second step we want to enhance this in a sophisticated representation, which moves the encountered changes (diffing on the server side, which optionally takes the stored rolling hashes into account) between two revision in an outer ring:

http://www.youtube.com/watch?v=l9CXXBkl5vI

https://github.com/JohannesLichtenberger/master-thesis/blob/master/Master/Thesis/thesis.pdf

Thus, the two revisions to be compared are merged into one tree or SunburstView.

docker-compose up -d frontend Error

I'm trying to set up the project following https://github.com/sirixdb/sirix-web-frontend#project-setup, when I run step 14, I get the following message:

Building frontend
ERROR: Cannot locate specified Dockerfile: Dockerfile

I am running docker toolbox on windows 10.

Using the new composition API

What do you think about using the new composition API from Vue.js?

Vue.js Auth Component

We need a component, which extracts the code from the redirected to URL (from the OAuth2 auth code flow) and exchange this code with a token (/token) endpoint on the SirixDB HTTP-Server. This token then needs to be sent in subsequent requests, so we can probably store it in the store!?

TableView / Change the JavaScript code to TypeScript code

We need to change the example JavaScript code to TypeScript code and then start work on getting the data from a running HTTP-Server via the Axios HTTP-Client for instance.

Side by side comparison view

I envision having two revisions of a resource or a subtree thereof (filtered be a query in the first place) side by side and showing the differences. This has the huge benefit that we do not do line-by-line diffs. Instead we are really diffing the tree-structures and thanks to rolling hashes optionally stored fod each node the diffing algorithm can skip whole subtrees from the comparison :-) furthermore I might add a very simple index (which node keys haved changed and how) if you compare consecutive revisions.

Setup some CI/CD tests

Have to read about how to test the frontend stuff... besides Unit Tests for some stuff.

@inject called with undefined this could mean that the class undefined has a circular dependency problem.

With most recent changes, I started seeing these errors in console:

7d8c56d89e132e2d7879.js:2 Uncaught (in promise) Error: @inject called with undefined this could mean that the class undefined has a circular dependency problem. You can use a LazyServiceIdentifer to  overcome this limitation.
    at 7d8c56d89e132e2d7879.js:2
    at 7d8c56d89e132e2d7879.js:2
    at 7d8c56d89e132e2d7879.js:2
    at Object.<anonymous> (7d8c56d89e132e2d7879.js:2)
    at o (7d8c56d89e132e2d7879.js:2)
    at Module.569 (2c4aaddaf3f9d5a3bfbb.js:1)
    at c (9a011b95eb207f07c163.js:1)

TreeView

Create a tree-view component which displays a JSON or XML file. The data comes from the database backend, which is able to store huge instances. Thus we might limit each level to a predefined number of nodes.

Furthermore the children should be loaded on demand from the backend, when a node with children is expanded in the tree-view. To query the children from this node we need a possibility to add the unique node identifier to the serialized JSON form in the first place (how to add this kind of metadata which should not be displayed in the views is up for discussion).