janejeon / blink

Modern, lightweight, planet-scale link shortener for teams 🎉

Welcome to Blink 👋

Modern, lightweight, planet-scale link shortener for teams 🎉

Easy to setup, connect with your org's SSO, hook up analytics, and extend!

See the documentation for more information about the project itself (including screenshots)

Development

Prerequisites

You need the following components to develop and run Blink locally:

• mkcert
• docker & docker-compose (note that on Linux, docker & docker-compose are two separate deps)
• an x86-based machine (unfortunately, Keycloak - even in its latest version - does not work with ARM-based devices)

Before you start

1. Run make cert to generate the SSL certs required for HTTPS local development

2. Run make up to stand up the containers

And once you're done with development, you can run make down to shut down and cleanup all the containers that spun up.

NOTE: that you actually do not have to run the cleanup every time you npm start; you can shut down the npm start server/frontend combo and re-boot it as many times as you'd like without needing to make up every time - the migrations and the build process will run fine even with existing data(!)

Starting Blink

Run make dev to stand up the development environment (i.e. it will run the actual container in which Blink will be run in "dev mode", which is fancy term for NODE_ENV=development). Then, run npm start to run Blink, which consists of a backend express server and a frontend react "live-loader". You can access the app at https://localhost/app! Both the frontend and the backend will live-reload as you make changes. You can login as the user specified at the end of config/keycloak.json (username: user, password: Password1).

Behind the scenes, the frontend (a create-react-app app) is running at http://localhost:4000/app and is being reverse proxied from https://localhost/app, and everything else in https://localhost gets proxied to the backend at http://localhost:3000. Yes, there are two processes running in the container pretending to be one "site", but this kind of routing (thanks Traefik!) allows us to not have horrible routing issues (stemming from the fact that even though they're both http://localhost, the different port means they're effectively two different sites and leads to a whole host of routing, CORS, and other integration issues), and allows testing of features that are only available for HTTPS in order to simulate real-world usage as much as possible.

Troubleshooting

If you can't reach any service or if you suspect the routing is messed up, first check https://traefik.localhost to make sure that everything is configured correctly.

Note that for performance/security reasons, the app container is built end-to-end with alpine-based images, so you won't have access to anything fancy like, say, bash.

Run tests

make test COMMAND='npm test'

# or, for interactive testing:
make test
$ npm run test:watch

Deployment with Docker Compose

Under the /Deploy/Docker-Compose/ directory in this repo is a base docker-compose.yml and .env files that you can use to deploy this with no building necessary. Just ensure the database details are entered correctly (strongly advise using a different password for the database than is currently specified) and that you've added relevent OIDC/OAUTH/SESSION config to the .env file and you should be good to go.

Simply download both files, and run:

docker compose up -d

Author

👤 Jane Jeon

• Github: @JaneJeon

🤝 Contributing

Contributions, issues and feature requests are welcome!
Feel free to check issues page.

Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2022 Jane Jeon.
This project is AGPL-3.0 licensed.

TL;DR: you are free to use this application "as-is" in your code or on its own, without needing to make your code source-available or to license it under the same license as this application; however, if you do change this application and you distribute it (which includes using it to provide a public service over the network), please do contribute back any improvements for this application.

---

This project is based off of Express-Objection-Template, and heavily uses objection-authorize and objection-hashid to drive much of "hidden" logic and make isomorphism possible.

blink's Issues

Migrate to postgres+redis

casl packages are annoying AF, the only reason we were using mongoose was to try casl.

figure out

Integrate SWR w/ react-admin to dynamically update state

Blocked on #29. Also see #169.

Plan out the UI (like draw it)

And plan ahead for use by both private and public organizations of various sizes.

And really do consider using stock solutions, like keystone or strapi or adminbro.

JSON Schema-based form validation on the frontend

Convert to objection models

Including existing features

Rename project to Lynx

Cause honestly "myURL" was a bad name...

jest called with react-scripts does not generate reports, wtf

https://github.com/JaneJeon/lynx/blob/8b248efc49883049fae11f234c2412b28155dbbb/.circleci/config.yml#L49-L53

---

This issue was generated by todo based on a TODO comment in 8b248ef. It's been assigned to @JaneJeon because they committed the code.

Alert when link dies

Preserve certain URLs

I'm just not sure how to go about this, or which ones to preserve.

For example, /login might be used internally to redirect people to the login page, etc.

Dockerize the entire stack

Right now I’m dockerizing basically everything except the actual node server. With the addition of workers, it would be helpful to dockerize the entire stack so development & testing becomes easier.

However, there are some challenges:

1. Making code/dependency changes transparent to the development/test container
2. Making sure I don’t recreate everything from scratch anytime I change anything
3. “Productionizing” the stack (esp. with regards to HTTPS and reverse proxy - might have to turn to Traefik, eww), though that may come later after I get the dev/test setup completely dockerized.

Also of note, this is blocked by #428 since Circle does NOT play nice with docker-compose testing...

JSON Schema-based form validation

Should come after #29

Write docs

do we need to stand up containers even for frontend?

https://github.com/JaneJeon/lynx/blob/29a450e11f8813579878b40bd17d3ed7458068a6/.circleci/config.yml#L40-L45

---

This issue was generated by todo based on a todo comment in 29a450e. It's been assigned to @JaneJeon because they committed the code.

Allow 👌emojis🤣💯

We just updated the upstream objection-hashid dependency, meaning we can include EmOjIs as part of the link shortener.

EMOJIS.

To do this, we need to LOCK DOWN the alphabet (because otherwise the hashid for existing links might break), meaning we can't just "add" emoji support later on. So we'd have to include ALL emojis to support in the JSON schema regex...

Frontend SSO integration

• Endpoints for getting user info from user id’s (have to call OIDC userinfo endpoint, good fucking luck configuring that oof)
• “hydrating” the frontend information with said user info (e.g. user name/avatar)
• login/logout
• “self” info (aka who you’re logged in as)

Migrate to GitHub Actions

Write tests for existing features

• User models
• User routes

Figure out what form of dashboard I want - SPA vs. SSR?

In either case, I want to renter the dashboard pages with React, just so I can try out @casl/react.

	Pros	Cons
SPA	Simpler (CRA); detox practice	Testing is hard due to ES6 import; subdomain needed; not sure how auth works; routing
SSR	integrates with express; Component-based routing; css-in-js	More complexity?

Migrate to objection v2

Make Blink MySQL/Postgres agnostic by using varchar

We should also test w/ both MySQL and Postgres databases...

Plus I’m also leaning towards yeeting the knex case mapper

Write basic e2e tests to make sure everything is plugged in well

This should preferably be done with the “production” build (aka static SPA being served from the backend) since I’m pretty sure I fucked up some URL config somewhere

Hash setter

Instead of directly setting id, maybe have hash setter? But how will this affect the ACL's property-based rules?

Also tangentially related to https://github.com/JaneJeon/myURL/projects/6?

Offset-based pagination instead of cursor (incl. UI)

Server-pushed events

E.g.

• rather than hitting the database every request, cache id => req.user and make a node instance listen to the postgres changes. When a user change comes through, bust that cache.
• rather than having to hit settings every request (see #143), cache settings and basically only bust it when the node is notified of a settings change.

This can probably also help with #158, as having a stream of changes makes it easy to save it.

Migrate policy to casl

role-acl works well for simpler stuff, but the need to rely on the clunky $. syntax for comparing anything and the inflexibility of constructing policies (it ALWAYS has to compare a resource field against a body field) is really shit for larger projects.

Audit log/notifications

https://vincit.github.io/objection.js/recipes/polymorphic-associations.html

Ditch Heroku for ansible/docker-compose deployment

Continuous Integration

https://dev.to/alexandrupero/continuous-delivery-with-github-actions-docker-and-traefik-on-a-virtual-private-server-part-1-3285 (i.e. build & push images to github docker registry)

Continuous Deployment

https://medium.com/swlh/deploying-docker-compose-applications-with-ansible-and-github-actions-7f1740392507
https://stackoverflow.com/questions/42302632/docker-compose-continuous-deployment-setup

Allow duplicates/changing when branding existing links

Specifically, when you have a link that has randomly generated hash and you want to give it a custom hash, either allow duplicate copies to exist or move the randomly generated one into the custom one

Just look at what bitly/kutt does and copy it I guess

Canonicize user-defined hashes

/blah and /BlAh should both lead to the same place. While we're at it, should we have this be case-insensitive?

Copy from objection-template

and convert ACL while I'm at it

Add GitLab integration

(and to be able to tie users between GH/GL means I'd need to normalize emails and use that as the sort of primary key)

Cache the entire node_modules

I just noticed that despite caching ~/.npm, CircleCI is still downloading Firefox and Chrome!

- restore_cache:
        keys:
          - v1-deps-{{ checksum "package-lock.json" }}
    - run: npm install
    - save_cache:
        key: v1-deps-{{ checksum "package-lock.json" }}
        paths:
          - node_modules

Migrate to ES Modules w/ node@>=13.2

Possibly move to restana

While /:hash doesn't work for find-my-way, you can have "default" route handlers... do we want to move to restana (not fastify because they don't believe in monkey patching and won't work with passport)?

add basic UI

• Page for POST /links
• Page for GET /links
• Page for GET /links/:id
• Page for GET /users
• Page for GET /users/:id
• Page for GET /users/:id/links
• plus whatever navigational tools are needed (e.g. navbar)

No analytics yet. Depends on #31.

Build the dashboard (ignore the SSO integration for now)

• Links page
• The link shortener main page? Preferably pack it into a component so it can be moved later down the line
• Make Link views refer to users' names based on their ID's (lookup IdP)

Convert routes to support new models

Move to github actions

For that sweet, sweet free runner minutes

Convert to snake_case in the database

• Add knexcasemapper
• Change originalURL to originalUrl, and so on. This needs to be done for JSON Schema, model properties, and the frontend

Access control integration (isomorphism!)

Advanced access control & revised policies

1. The policy itself should be configurable by the owner/admin
2. Adding a hash/setting a hash/"ownership" transfer after a user's deletion, etc.

Add note about this being self-hosted

Make nextjs server stop on ctrl+C

At the very least, when the express server stops, so should the nextjs server (webpack for dev, not sure for prod)

Take out --forceExit

See knex/knex#2686

Transaction-ify tests

Currently, a few tests are hacky (at best) due to having to worry about global state during tests.

I should "transaction-ify" tests to make sure every test is idempotent.

Dashboard

We need some sort of dashboard to manage links and view stats.

Some candidates/inspirations:

• custom
• keystone
• strapi (imo too much scope)
• adminbro
• https://administrate-prototype.herokuapp.com/admin (holy shit it looks beautiful)

Built-in analytics based on CloudFront logs

This would be the "default" analytics, but it should be pluggable.

Change wording around serverless

And point out the new focus

Implement access control on the frontend

Use acl.yml