GithubHelp home page GithubHelp logo

ethyca / fidesdemo Goto Github PK

View Code? Open in Web Editor NEW
5.0 5.0 3.0 1.27 MB

Standalone project demonstrating the use of Fides in an example Flask application

License: Apache License 2.0

Makefile 11.45% Python 68.66% CSS 1.99% HTML 13.76% Shell 2.87% Dockerfile 1.27%

fidesdemo's Introduction

Fides Demo Project

This is a basic Flask app that demonstrates the use of fidesctl and fidesops as part of a "real" project that uses:

  • Flask to run a web server simulating a basic e-commerce application
  • PostgreSQL as an application database
  • SQLAlchemy to connect to the database
  • fidesctl to declare privacy manifests and evaluate policies
  • fidesops to execute privacy requests against the Flaskr database

The Flask app itself is simply the Flask tutorial app modified to simulate an e-commerce marketplace, in order to highlight some basic examples of data categories that might be stored in a "real" user-facing application.

Requirements

This demo project is currently only supported on Mac OS, as the Makefile uses shell commands that aren't available on Windows.

To run this project, first ensure you have the following requirements installed and running on your machine:

  • Docker 12+
  • Python 3.7+
  • Make
  • pg_config (on Mac, install via brew install libpq or brew install postgres)

Getting Started

To create the project venv and install requirements, run:

make install

Once installed you can run the full demo environment with:

make demo

After a few seconds, this will open several browser tabs to the various services. Read the terminal output for more information!

Run an example fidesops privacy request with:

make fidesops-request

Run an example fidesctl privacy evaluation with:

make fidesctl-evaluate

Example Application: "Flaskr"

This example application is meant to simulate a basic e-commerce marketplace where users can register and purchase products from each other. Using the website you can:

  • Register a new user
  • Login as a user
  • Post a "product"
  • Delete/update products you posted
  • Submit a purchase to a product

The schema itself is designed to highlight a few very simple examples of how identifiable data might get stored in even a trivial web application like this. The sample data below shows what this looks like:

flaskr=# SELECT * FROM users;
 id |     created_at      |       email       |              password              | first_name | last_name 
----+---------------------+-------------------+------------------------------------+------------+-----------
  1 | 2020-01-01 00:00:00 | [email protected] | pbkdf2:sha256:260000$O87nanbSkl... | Admin      | User
  2 | 2020-01-03 00:00:00 | [email protected]  | pbkdf2:sha256:260000$PGcBy5NzZe... | Example    | User
(2 rows)

flaskr=# SELECT * FROM products;
 id |     created_at      | seller_id |       name        |             description              | price 
----+---------------------+-----------+-------------------+--------------------------------------+-------
  1 | 2020-01-01 12:00:00 |         1 | Example Product 1 | A description for example product #1 |    10
  2 | 2020-01-02 12:00:00 |         1 | Example Product 2 | A description for example product #2 |    20
  3 | 2020-01-03 12:00:00 |         2 | Example Product 3 | A description for example product #3 |    50
(3 rows)

flaskr=# SELECT * FROM purchases;
 id |     created_at      | product_id | buyer_id |    street_1    | street_2 |    city     | state |  zip  
----+---------------------+------------+----------+----------------+----------+-------------+-------+-------
  1 | 2020-01-04 12:00:00 |          1 |        2 | 123 Example St | Apt 123  | Exampletown | NY    | 12345
(1 row)

License

This project is licensed under the Apache Software License Version 2.0.

fidesdemo's People

Contributors

eastandwestwind avatar nevilles avatar seanpreston avatar thomaslapiana avatar galvana avatar conceptualshark avatar kepae avatar

Stargazers

 avatar Steve Murphy avatar  avatar Hao avatar Maria Patterson avatar

Watchers

Cillian avatar Steven Benjamin avatar  avatar  avatar Sachin Wagh avatar

fidesdemo's Issues

Update fidesops version to 1.8.2

A newer version of fidesops (1.8.2) has recently been released; the version in use by this repository is out of date.

Begin by updating the version in requirements.txt:

fidesops==1.8.2
๐Ÿ“ Note The version of fidesops must be pinned to an exact version number.

There may be additional changes needed in this repository to guarantee the inclusion of upstream feature enhancements, bug fixes, and/or to maintain compatibility. Be sure to review the changelog to ensure that any such code changes are made in tandem with the version bump.

Validate (and update) Fidesdemo for fidesctl 1.8.1

Is your feature request related to a specific problem?

fidesctl==1.8.1 was released. With that in mind, we should validate the demo workflow as is, and update accordingly.

This issue should focus on just the current functionality, not adding new content

Describe the solution you'd like

Walk through the fidesctl demo portion as is, and ensure no issues are present.

Describe alternatives you've considered, if any

Also including new content, but that seems like it would distract from the goal of ensuring the fidesdemo is fully functional

Additional context

https://github.com/ethyca/fides/releases/tag/1.8.1

Acceptance Criteria

  • The UI should load on the main branch of the demo

Update fidesops version to 1.8.0

A newer version of fidesops (1.8.0) has recently been released; the version in use by this repository is out of date.

Begin by updating the version in requirements.txt:

fidesops==1.8.0
๐Ÿ“ Note The version of fidesops must be pinned to an exact version number.

There may be additional changes needed in this repository to guarantee the inclusion of upstream feature enhancements, bug fixes, and/or to maintain compatibility. Be sure to review the changelog to ensure that any such code changes are made in tandem with the version bump.

`make clean` is overly aggressive; prunes all docker resources on system

Is your feature request related to a specific problem?

The make clean invocation is a bit aggressive; the current clean will execute docker system prune -a --volumes, which will clear all resources and volumes in the docker storage-- including those unrelated to fidesdemo. The command does require user input to proceed, but given the small amount of data in the postgres DB volume that this demo can generate it seems like a dangerous default clean.

Further, docker prune is executed first, before cleaning venv and fides-related artifacts. Cancelling the docker prune (like I did) then leaves these artifacts, which are perhaps more important to dispose of for the purpose of the demo.

Describe the solution you'd like

make clean should not prune volumes, and prompt the user to delete them themselves; it's better to leave a dangling volume than delete more than was installed by fidesdemo. (Which may happen with the prune anyways.)
Also, venv and fides-related cleaning should happen first, regardless of whether or the docker prune behavior should change. This way, users can opt-out of the docker-prune and still have the other files deleted.

In fact, make reset-db already calls docker volume rm fidesdemo_postgres, which should perhaps be the behavior replicated in make clean.

Describe alternatives you've considered, if any

One way to better target docker prune may be to use the --filter=<key>=<value> argument, if labels can be set in the docker-compose workflow for all docker resources created for the demo (e.g. containers, networks, volumes).

Update fidesops version to 1.8.1

A newer version of fidesops (1.8.1) has recently been released; the version in use by this repository is out of date.

Begin by updating the version in requirements.txt:

fidesops==1.8.1
๐Ÿ“ Note The version of fidesops must be pinned to an exact version number.

There may be additional changes needed in this repository to guarantee the inclusion of upstream feature enhancements, bug fixes, and/or to maintain compatibility. Be sure to review the changelog to ensure that any such code changes are made in tandem with the version bump.

Virtual Environment Activation Unclear

Bug Description

After running make install a comment states "Done! Run '. venv/bin/activate' to activate venv"

This could be made more clear to a potential user

Steps to Reproduce

Run make install, see comment

Expected behavior

The virtual environment should either be automatically created or noted as a step in the docs -> https://ethyca.github.io/fides/1.7.0/tutorial/

Screenshots

n/a

Environment

  • Version: 1.7.0
  • OS: Darwin
  • Python Version: 3.9.12
  • Docker Version:

Additional context

Will add more as needed potentially

Update fidesops version to 1.7.1

A newer version of fidesops (1.7.1) has recently been released; the version in use by this repository is out of date.

Begin by updating the version in requirements.txt:

fidesops==1.7.1
๐Ÿ“ Note The version of fidesops must be pinned to an exact version number.

There may be additional changes needed in this repository to guarantee the inclusion of upstream feature enhancements, bug fixes, and/or to maintain compatibility. Be sure to review the changelog to ensure that any such code changes are made in tandem with the version bump.

Improve Windows Support

Bug Description

Many of the make targets leverage some handy shell techniques that only work on UNIX systems, but I'd like to get this demo project working smoothly on Windows.

Steps to Reproduce

Attempt to run the minimal demo targets on Windows:

  1. make install
  2. make demo

Expected behavior

These commands should run successfully and the demo environment should be fully usable.

Update fidesops version to 1.7.2

A newer version of fidesops (1.7.2) has recently been released; the version in use by this repository is out of date.

Begin by updating the version in requirements.txt:

fidesops==1.7.2
๐Ÿ“ Note The version of fidesops must be pinned to an exact version number.

There may be additional changes needed in this repository to guarantee the inclusion of upstream feature enhancements, bug fixes, and/or to maintain compatibility. Be sure to review the changelog to ensure that any such code changes are made in tandem with the version bump.

make install gives error related to fidesctl distribution not available

Bug Description

A description of what the bug is.

Steps to Reproduce

I have setup the pre-requisites as required for fidesdemo.
When I execute the make install command, it gives me the below error while building the environment.

waghsac@WAGHSAC:~/fidesdemo$ make install
Bringing up docker containers...
WARN[0000] Found orphan containers ([fidesdemo-fidesops-1 fidesdemo-redis-1]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
[+] Running 4/4
โ ฟ Container fidesdemo-db-1 Started 1.2s
โ ฟ Container fidesdemo-fidesctl-1 Started 2.6s
โ ฟ Container fidesdemo-redis-1 Started 1.0s
โ ฟ Container fidesdemo-fidesops-1 Started 1.3s
localhost:5432 - accepting connections
Creating virtual environment ./venv...
Installing project dependencies...
Collecting black>=21.9b0 (from -r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/2e/ef/a38a2189959246543e60859fb65bd3143129f6d18dfc7bcdd79217f81ca2/black-22.3.0-py3-none-any.whl
Collecting click>=7.0.0 (from -r requirements.txt (line 2))
Using cached https://files.pythonhosted.org/packages/4a/a8/0b2ced25639fb20cc1c9784de90a8c25f9504a7f18cd8b5397bd61696d7d/click-8.0.4-py3-none-any.whl
Collecting fidesctl==1.2.0 (from -r requirements.txt (line 3))
Could not find a version that satisfies the requirement fidesctl==1.2.0 (from -r requirements.txt (line 3)) (from versions: )
No matching distribution found for fidesctl==1.2.0 (from -r requirements.txt (line 3))
Makefile:32: recipe for target 'install' failed
make: *** [install] Error 1

I'm using Ubuntu 18.04 LTS on Windows.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.