priyank-purohit / postgui Goto Github PK

View Code? Open in Web Editor NEW

428.0 15.0 51.0 5.34 MB

A React web application to query and share any PostgreSQL database.

License: GNU General Public License v3.0

JavaScript 84.31% HTML 0.24% CSS 1.97% PLpgSQL 2.04% TypeScript 11.44%

reactjs postgresql postgres postgrest gui dashboard bioinformatics genomics data-sharing database

postgui's Introduction

PostGUI

A React web application to query and share any PostgreSQL database.

Current Status ✅ ✅

This app is currently being updated in the v2 branch. This new version is written in Typescript, and follows a set of modern React development standards.

Introduction
Features
Use Case Scenarios
Installation
Security and Deployment Checklist
Contact

Introduction

PostGUI is a React web application that serves as a front-end to any PostgreSQL database using the PostgREST automatic API tool. PostGUI can automatically adapt any PostgreSQL database to provide an overview of the schema, a visual query builder to execute a SQL query, and a login system to control access to the data. The entire framework can run on a personal computer, or on a web server (for data sharing over the internet).

This is a Masters project intended to be a web-based framework for data sharing and making data more accessible to non-computer programmers. Originally intended for biology community, the generic approach towards data has made this framework into a simple alternative to creating a new custom web application each time data set(s) need to be shared. These web applications are usually non-adaptive and require significant changes if the data source changes.

A PostgreSQL database is a pre-requisite to setting up this framework - specific tips on database schema design are provided in the Installation section. Configuration of PostgREST and PostGUI can be done in a matter of minutes. The PostGUI configuration file (at /data/config.json) in this repository demonstrates how to customize the front-end for a sample PostgreSQL database. PostgREST configuration is provided, along with some suggestions on PostgreSQL database schema design for optimal experience.

To get help with confirming use cases/feasibility or anything else, please make a new issue on this GitHub repository. This data sharing framework relies on popular open-source works such as: Material UI, PostgREST, JS Query Builder, etc.

Features

Data Compatibility

PostGUI is designed to be database agnostic by automatically adapting to the client's database schema. This framework can be setup to query and share a biology, finance, or a sports statistics PostgreSQL database. In short, any tabular data set can be accessed and shared using this framework. If the data set(s) in question can be represented as tables, then the data set(s) is compatible with this framework. Complex data types such as images, videos, or sound clips are currently untested and unsupported.

The client must organize the data to be shared in a PostgreSQL database running on a local computer (only data access, no data sharing over the internet) or a web server (data access, and data sharing over the internet). Certain features of PostGUI require specific database schema design. For example, to allow users to edit the database tables, the database tables must have a defined primary key (PK) attribute. However, if a PK is not defined for a table, PostGUI will gracefully disable the edit feature for the table.

Database Picker

The database picker allows multiple PostgreSQL databases to be shared from a single instance of PostGUI. The database name is customizable using the /data/config.json file.

Database Schema

The database schema (tables, columns, and foreign keys) is shown in the left panel of the PostGUI user interface (UI). The tables and columns can be renamed for a more user-friendly table and column name. Foreign Keys also are shown with “Referenced by” and “FK to” labels.

Search Feature

Search feature can be used to find a table or column quickly. To search for a specific table vs. a column, each search term can be tagged with ‘[table]’ or ‘[column]’.

Query Builder and Query Options

Integration of the JS Query Builder in this web application makes it easily usable by users unfamiliar with SQL programming language. Query options can be used to fine tune the data being extracted from the database, and to ensure the full result is being shown (exact row count feature).

Data Table

The query result component features a high-performance data table (React Table) that can sort columns by their values and edit individual cell values (if edit feature is enabled).

Download Data

The Downloads card features options to download the currently loaded data in CSV, JSON, and XML formats. In addition, it allows for column values to be copied as comma-separated values – this may be used in lieu of a join-table VIEW.

Authentication System

Basic authentication system gets you started with the “database-first” approach to secure authentication system. Three basic users are available by simply executing the Authentication SQL script – read, edit, and admin. The Installation section provides further instructions on how to set up the authentication system. Further comments in the Security Checklist section.

Edit Contents

Edit Content feature allows authenticated users to change the table contents if a primary key is defined for the table. A record of database changes is also kept in a separate table.

Use Case Scenarios

This project was built to solve three main scenarios:

Accessing a database (no data sharing)

Set up the PostgreSQL database, PostgREST API tool, and the PostGUI web application on a local computer for database access without any data sharing over the internet.

Sharing a database (read-only data sharing mode)

Set up the PostgreSQL database, PostgREST API tool, and the PostGUI web application on a web server for data sharing over the internet.

Sharing a database with edit access (read and write access for authenticated users)

Set up the PostgreSQL database, PostgREST API tool, and the PostGUI web application on a web server. In addition, enable the edit feature by ensuring each table has a primary key defined, presence of the primary_keys function in the PostgreSQL database, and enabling the feature in the configuration file of PostGUI.

Installation

Pre-requisite

This installation guide will walk you through setting up a production-like version of the framework with all features enabled on your personal computer. After testing on your personal computer, deploying the three components of the framework on a web server will allow you to share the data over the internet.

Get Software

Install Node.js.
Install PostgreSQL.
Download PostgREST.

PostgreSQL Setup

Get Started with a Sample Database

Download the DVD Rentals database file from PostgreSQL Tutorial website.
Unzip the file to retrieve the .tar file.
Follow these instructions using a command line tool: http://www.postgresqltutorial.com/load-postgresql-sample-database/.

Get Started with Your Data

This framework is compatible with any data set(s) that can be represented as a data table. The data set(s) is required to be inserted into the PostgreSQL database system to ensure data integrity and consistent data access. Having a well designed PostgreSQL database is important because PostGUI does not enforce any hard requirements on the database schema design. However, to take full advantage of all PostGUI features, do consider the following when designing your PostgreSQL database schema:

Ensure necessary column and table attributes are present:
1. Foreign Key: required only to show two-way relationships in the left panel in addition to the foreign_key function from the /scripts/ directory.
2. Primary Key: required only to enable the Edit Feature in addition to the primary_key function from the /scripts directory.
3. NOT NULL, CHECK, and data types should be carefully considered because they serve as back-end sanitation of input coming from the Edit Feature.
Create necessary VIEWs: To prevent Denial or Service attacks, the PostgREST framework does not allow for arbitrary table joins. Therefore, the client must predefine join tables when queries spanning multiple tables will be executed frequently.
Create necessary indexes: important feature of large tables in order to speed up the query response time. When the CONTAINS operator will be used frequently with a string column, a pg_trgm idnex would be important to improve performance.

PostgREST Setup:

Download and extract the PostgREST binary file into a folder from: https://github.com/begriffs/postgrest/releases/tag/v0.4.4.0.
Create a configuration file named db.conf in the same folder as the PostgREST binary.

db.conf file should contain the following:

     db-uri = "postgres://DB_USER:DB_USER_PASS@localhost:5432/DB_NAME"
     db-schema = "public"
     db-anon-role = "DB_USER"
     db-pool = 10
     server-host = "*4"
     server-port = 3001

This configuration file can be further customized as described in the original docs: https://postgrest.com/en/v4.4/install.html#configuration.

Be sure to replace DB_NAME, DB_USER, DB_USER_PASS in the above configuration file.
Using a command line tool, navigate to the folder containing the PostgREST binary and config file.
Run the PostgREST tool: ./postgrest.exe ./db.conf (remove the .exe extension if not on Windows operating system).
You should see a "Connection successful" message.

PostGUI Setup:

Open a new command line terminal.
Clone this repository: git clone [email protected]:priyank-purohit/PostGUI.git
Navigate into the cloned repository: cd PostGUI.
Install node dependencies by executing: npm install.
Create a basic configuration file for the front-end web application:
1. Configuration file location: /src/data/config.json.
2. Change the url if any changes were made to the PostgREST port, or if the PostgREST instance is running on a web server.
3. Give an appropriate title to the database.
Using the command line tool, execute npm start to run the web application.
Improve configuration file for better user experience according to the Config File Format section of this guide.

Security and Deployment Checklist

Before deploying the framework on a web server for, consider the following items to improve security.

Harden PostgREST for improved performance and security.
1. Go through the PostgREST docs.
2. Go through https://postgrest.org/en/v5.0/admin.html section.
Enable HTTPS for PostGUI
1. Install SSL/TLS certificate on the web server. This should automatically enable HTTPS for the web application.
2. Force the web server to handle the PostgREST traffic as HTTPS by setting up a reverse proxy on the port used by PostgREST.
  1. A sample NGINX configuration file is provided in this repository at /scripts/Sample NGINX Config File.txt.
Authentication System
1. Be sure to enable HTTPS first.
2. Execute the scripts in the /scripts/ directory to create an authentication system schema in the database.
3. Insert new emails and passwords for more users.
4. Fine tune authentication control according to the Authentication System section of this guide.

Contact

Priyank K. Purohit, Dr. David Guttman, Dr. Nicholas Provart

University of Toronto

postgui's People

Contributors

Stargazers

Watchers

postgui's Issues

Trying to npm start PostGUI on Windows 10 fails

I've downloaded a fresh .zip file and installed it according to the documentation. PostgREST is running locally on port 3020. A npm start will result in the following .log:

0 info it worked if it ends with ok
1 verbose cli [ 'C:\Program Files\nodejs\node.exe',
1 verbose cli 'C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js',
1 verbose cli 'start' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info lifecycle [email protected]prestart: [email protected]
6 info lifecycle [email protected]start: [email protected]
7 verbose lifecycle [email protected]start: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]start: PATH: C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin;c:\sources\post-gui\node_modules.bin;C:\Python37\Scripts;C:\Python37;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files (x86)\Intel\iCLS Client;C:\Program Files\Intel\iCLS Client;C:\ProgramData\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\WINDOWS\System32\OpenSSH;C:\Program Files\Intel\WiFi\bin;C:\Program Files\Common Files\Intel\WirelessCommon;C:\Program Files\nodejs;C:\Program Files\PostgreSQL\10\bin;C:\Users\Heinz-Peter Scharf\AppData\Local\Microsoft\WindowsApps;C:\Program Files\Intel\WiFi\bin;C:\Program Files\Common Files\Intel\WirelessCommon;C:\Program Files\Java\jdk1.8.0_144;%JAVA_HOME\bin;C:\bin\wildfly-10.1.0.Final;%JBOSS_HOME\bin;C:\Program Files\apache-maven-3.6.1\bin;C:\Users\Heinz-Peter Scharf\AppData\Local\Programs\Microsoft VS Code\bin;C:\Users\Heinz-Peter Scharf\AppData\Roaming\npm
9 verbose lifecycle [email protected]start: CWD: c:\sources\post-gui
10 silly lifecycle [email protected]start: Args: [ '/d /s /c', 'PORT=8771 react-scripts start' ]
11 silly lifecycle [email protected]start: Returned: code: 1 signal: null
12 info lifecycle [email protected]start: Failed to exec start script
13 verbose stack Error: [email protected] start: PORT=8771 react-scripts start
13 verbose stack Exit status 1
13 verbose stack at EventEmitter. (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\index.js:301:16)
13 verbose stack at EventEmitter.emit (events.js:198:13)
13 verbose stack at ChildProcess. (C:\Program Files\nodejs\node_modules\npm\node_modules\npm-lifecycle\lib\spawn.js:55:14)
13 verbose stack at ChildProcess.emit (events.js:198:13)
13 verbose stack at maybeClose (internal/child_process.js:982:16)
13 verbose stack at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
14 verbose pkgid [email protected]
15 verbose cwd c:\sources\post-gui
16 verbose Windows_NT 10.0.17134
17 verbose argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "start"
18 verbose node v10.16.0
19 verbose npm v6.9.0
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] start: PORT=8771 react-scripts start
22 error Exit status 1
23 error Failed at the [email protected] start script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

Could you please help to clarify why PostGUI doesn't start properly.

Thanks in advance!

pgjwt

Which version of the extension do you need for this? How can I go about installing it.

Connect to existing postgrest

Hi,
I have an existing RDS postgres database (AWS RDS) and a postgREST api running in AWS EKS.
I´d like to deploy this webUI in EKS as well, however did not understood where should i configure the url of the existing postgREST as url field seems to stand for the address/port to bind the webui and not the connection to api itself.

url: used by PostGUI to make network requests. The port should be different from the port used in the PostgREST configuration file

Can you provide some guidance on how to configure PostGUI with already existing postgREST API and postgres RDS?

Thanks

rpc/primary_keys and rpc/foreign_keys response

I'm not able to set the create/edit mode for my database tables. Is there any documentation in order to understand the configuration or responses that have to be made in these methods?

Unauthorized access

POST http://localhost:3001/rpc/login 401 (Unauthorized)

Authentication jwt and PostGUI

Hi all,

I am trying to setup a PostGUI (https://github.com/priyank-purohit/PostGUI) which makes use of postgrest but getting struggled with jwt authentication.

Does anyone here has some kind of experience with this app? I did the config as mentioned in POSTGUI read me but when i click on login button nothing happens... at postgrest logs i can see that the post /rpc/login succeeds but at postgui application level nothing happens (and there are no logs!).

Example of a manual post to /rpc/login:

TRUT10236:network-catalog-ui-api bruno.gante$ curl -X 'POST' 'http://127.0.0.1:3000/rpc/login' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{"email": "[email protected]", "pass": "CHNAGME123"}'
{"token":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiZWRpdHVzZXIiLCJlbWFpbCI6ImFkbWluQGRvbWFpbi5jb20iLCJleHAiOjE2NjIxMjMwNjd9.INmxG3p0k5rFvOqfzh0pqsNmUGw6TYaZvJPAWTXeT3o"}
Example of postgrest log when i try to login in PostGUI with valid credentials:

172.18.0.1 - - [02/Sep/2022:11:38:53 +0000] "POST /rpc/login HTTP/1.1" 200 - "http://localhost:8771/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"

Example of postgrest log when i try to login in PostGUI with invalid credentials:

172.18.0.1 - - [02/Sep/2022:11:59:22 +0000] "POST /rpc/login HTTP/1.1" 403 - "http://localhost:8771/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"

Is there someone here that could help please? @priyank-purohit @erjanmx