dailybruin / sources

A website to manage news sources.

Sources

A website the Daily Bruin uses to keep track of all of our sources. It's powered by an Express server with a PostgreSQL backend and a GraphQL endpoint, and uses React on the frontend.

The production version is only available to Daily Bruin members.

Technologies Used

Below is a non-exhaustive list of some of the more major technologies used by Sources and the reasoning why these technologies were chosen. This list is meant to give you a starting point for considering adding or changing the technologies used by Sources or other projects and should change over time as needs and technologies change.

General

• TypeScript is a language developed by Microsoft. It's like JavaScript with types!
• Jest is testing framework.
• Prettier is an code formatter that makes sure all our code looks pretty!

Frontend

• React is a popular JavaScript library for building user interfaces.
• Parcel is an application bundler focused on simplicity and speed. It's very new and currently has a lot of bugs and limitations but shows a lot of promise and is much easier to use than alternatives such as Webpack.
• Apollo Client is a popular GraphQL client that works very nicely with React.
• react-table is a small and extensible component for displaying data (in our case, all of the Sources). It was selected because it's popular and had all the features we need. A potential alternative that was considered was react-data-grid, but we liked the lightweightness and styling of react-table better.
• react-contextmenu is a simple popup context menu used to display options to edit and delete sources. We actually started by using react-contextify, but switched to react-contextmenu as it was more popular and appeared to be more actively developed.
• react-modal is a simple modal component used to display the "Add" and "Edit" modals. A couple of modal components were considered, but react-modal seemed to be the least opinionated and did not come bundled as part of a larger library, such as react-bootstrap.

Backend

• Node almost needs no introduction. A JavaScript runtime for servers.
• Express is the most popular web framework for Node.
• Google APIs Node.js Client are used to authenticated users via Google Oauth 2.
• Apollo Server is a popular serverside package for creating a server that supports GraphQL queries.
• Sequelize is an ORM (Object-Relation Mapping) library for SQL in JavaScript. Essentially, it allows us to represent our database in JavaScript. Pretty nifty!

Services Used

In addition to the technologies used about, Sources has a few services integrated with the repository to make maintenance and development easier. They are:

• Travis CI for continuous integration.
• CodeCov for code coverage visualization.
• dependencies.io for dependency management and automatic updates.
• Code Climate for code analysis.

Structure

.
├── LICENSE
├── bin
│   └── www                     # The file the actually runs the node server.
├── coverage/                   # Folder generated by Jest for use with CodeCov.
├── dependencies.yml            # File used by dependencies.io.
├── designs/                    # Design documents and mockups
│   ├── flowcharts
│   ├── flowcharts.sketch
│   ├── mockups
│   ├── mockups.sketch
│   ├── site-flow.png
│   └── spec.md
├── dist/                       # Autogenerated folder where compiled TypeScript (i.e., JavaScript) code is.
├── package.json                # Project and Dependency metadata
├── readme.md                   # This readme!
├── scripts/                    #
│   └── create-data.ts
├── src
│   ├── app.ts                  # The main Express server.
│   ├── controllers             # Where controllers go
│   ├── errorHandling.ts        # Code for server error handling (e.g., 404, 500, etc.)
│   ├── models/                 # Where models are stored
│   ├── routes/                 # Routing logic goes here. Contains URL endpoints as well as GraphQL schemas.
│   └── views/                  # Where views are stored
├── tsconfig.json
├── tsconfig.server.json
├── tsconfig.test.json
├── tslint.json                 # TSLint configuration.
└── yarn.lock                   # Autogenerated file by Yarn.

Sources follows an MVC pattern.

The models are defined through Sequelize and stored in the models folder, along with some general database configuration. Currently there's only one model, Source.

Controllers are stored in the controllers folder. There are 2 controllers, authController and sourceController that take care of Google Oauth authentication and interfacing with the Source model respectively.

Frontend files are stored in the views folder. This includes both the login and main pages (login.html and index.html respectively) as well as React components (in the components/ directory).

Note that test files are put in the same directory as the respective file they test and have the extension .test.ts.

Contributing to Sources

Prerequisites

There are a couple of programs and files Sources depends on that you'll need to have installed to run it locally.

1. Node – You can install this with a simple brew install node on macOS.
2. Yarn – Yarn is a JavaScript package manager, and an alternative to npm. We prefer it to npm because it has a couple of nice features such as caching, lockfiles, and faster downloads.
3. Visual Studio Code – VS Code is an open source text editor built by Microsoft and has amazing tooling support for TypeScript projects (as well as a lot of other awesome features!). You can use any text editor you want, but VS Code will give you a nice experience :).
4. PostgreSQL – Postgres is the database we use. Installing it locally can be a little daunting, so check out our guide below if you're unfamiliar with it!
5. A .env file – It's good practice to store things like API keys and database information with environment variables so that they can easily be changed for different environments without code changes. This is cool, but when you have a lot of configuration variables it can become hard to manage, so we use a package called dotenv that reads configuration data from a file called .env and puts them into process.env. Check out our guide below for an example .env.

Installing PostgreSQL

Mac

You want to install Postgres through Homebrew.

brew install postgres

Then start Postgres with the command:

brew services start postgresql

Awesome! For reference, you can stop postgres at any time with berw services stop postgresql. You don't want to do that right now, though! Instead, you'll want to create Sources' database. We call ours sources.

createdb sources

You'll also want to create a dummy database used for testing. It's called sources-test.

createdb sources

Note that by default, the Postgres user is your computer's username and there is no password. When creating your .env file, the values will look something like:

DATABASE_HOST=localhost
DATABASE_USER=nathan
DATABASE_PASSWORD=
...
(other values)
...

Dope. Now you'll want to just create some data with yarn create-data and you're ready to go!

For development, you may also find it useful to use a GUI client to visualize the database. There a lot of great options, but we recommend PSequel!

Windows

We're working on this! If you know how to install Postgres on Windows, please make a pull request!

Example .env File

If you're making your own, the values you'll want are:

DATABASE_HOST=
DATABASE_USER=
DATABASE_PASSWORD=
G_CLIENT_ID=
G_CLIENT_SECRET=

sources's Issues

Use DataLoader

https://www.howtographql.com/graphql-js/7-using-data-loaders/

Sources crash procedure

Check our wiki page, there are some scripts you need to run

Setup CI/CD pipeline

It'd be cool if whatever was in master was what was live. DB uses docker and rancher to deploy and this book looks like a good starting point on setting up CI/CD?

Note we use Travis CI instead of Jenkins.

Add CI/CD to sources

Let's do itttt

Create Function add Declare Variable.

Hi,

I want to append some declare variable in between $BODY$ and BEGIN tag for Postgresql Function.

Thanks

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• Replace dependency cpx with cpx2 2.0.0
• Replace dependency faker with @faker-js/faker
• Update Node.js to a681bf7
• Update postgres:latest Docker digest to 6b841c8
• Update Node.js to v20
• Update dependency concurrently to v8
• Update dependency css-loader to v6
• Update dependency dotenv to v16
• Update dependency faker to v6 (faker, @types/faker)
• Update dependency graphql to v16
• Update dependency graphql-tools to v9
• Update dependency husky to v9
• Update dependency lint-staged to v15
• Update dependency match-sorter to v6
• Update dependency nodemon to v3
• Update dependency pg to v8
• Update dependency prettier to v3
• Update dependency react-table to v7
• Update dependency stylelint to v16
• Update dependency supertest to v6 (supertest, @types/supertest)
• Update dependency ts-node to v10
• Update dependency typescript to v5
• Update react monorepo to v18 (major) (@types/react, @types/react-dom, react, react-dom)
• 🔐 Create all rate-limited PRs at once 🔐

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• Update dependency codecov to v3.7.1 [SECURITY]

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update dependency debug to v4.3.1 [SECURITY]
• Update dependency passport to v0.6.0 [SECURITY]
• Update dependency apollo-server-express to v2 [SECURITY]
• Update dependency googleapis to v39 [SECURITY]
• Update dependency knex to v2 [SECURITY]
• Update dependency node-sass to v7 [SECURITY]
• Update all non-major dependencies (@types/cookie-parser, @types/cpx, @types/express, @types/faker, @types/graphql, @types/jest, @types/knex, @types/morgan, @types/react, @types/react-dom, @types/serve-favicon, @types/supertest, apollo-cache-inmemory, apollo-client, apollo-link-http, body-parser, concurrently, cookie-parser, dotenv, express, express-session, global, graphql, graphql-tag, graphql-tools, husky, lint-staged, morgan, nodemon, parcel-bundler, pg, prettier, prop-types, react, react-apollo, react-contextmenu, react-dom, react-modal, react-table, stylelint, supertest, ts-jest, tslint, tslint-config-prettier, typescript)
• Update Apollo GraphQL packages (major) (apollo-client, react-apollo)
• Update dependency @types/dotenv to v6
• Update dependency passport-google-oauth to v2
• Update dependency tslint-microsoft-contrib to v6
• Update dependency tslint-react to v4
• Update jest monorepo to v29 (major) (@types/jest, jest, ts-jest)
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

docker-compose

docker-compose.yml

• postgres latest@sha256:e6708d2202efbeada699821bafb710365d6d13d61ab5d838a015d3acd08d6f7e

dockerfile

Dockerfile

• node carbon@sha256:7b65413af120ec5328077775022c78101f103258a1876ec2f83890bce416e896

npm

package.json

• @types/cpx 1.5.0
• @types/knex 0.15.0
• @types/serve-favicon 2.2.30
• apollo-cache-inmemory 1.3.9
• apollo-client 1.9.2
• apollo-link-http 1.5.5
• apollo-server-express 1.3.6
• body-parser 1.18.3
• concurrently 4.0.1
• cookie-parser 1.4.3
• cpx 1.5.0
• debug 4.1.0
• dotenv 6.1.0
• express 4.16.4
• express-session 1.15.6
• faker 4.1.0
• glamor 2.20.40
• glamorous 4.13.1
• global 4.3.2
• googleapis 34.0.0
• graphql 14.0.2
• graphql-tag 2.10.0
• graphql-tools 4.0.3
• knex 0.15.2
• match-sorter 2.3.0
• morgan 1.9.1
• passport 0.4.0
• passport-google-oauth 1.0.0
• pg 7.6.1
• prop-types 15.6.2
• react 16.6.3
• react-apollo 2.2.4
• react-contextmenu 2.10.0
• react-dom 16.6.3
• react-modal 3.6.1
• react-table 6.8.6
• serve-favicon 2.5.0
• @types/cookie-parser 1.4.1
• @types/dotenv 4.0.3
• @types/express 4.16.0
• @types/faker 4.1.4
• @types/graphql 14.0.3
• @types/jest 23.3.9
• @types/morgan 1.7.35
• @types/react 16.7.4
• @types/react-dom 16.0.9
• @types/supertest 2.0.6
• babel-core 6.26.3
• babel-preset-env 1.7.0
• babel-preset-es2015 6.24.1
• babel-preset-react 6.24.1
• babel-preset-stage-2 6.24.1
• codecov 3.1.0
• css-loader 1.0.1
• husky 1.1.4
• jest 23.6.0
• lint-staged 8.0.4
• node-sass 4.10.0
• nodemon 1.18.6
• parcel-bundler 1.10.3
• prettier 1.15.2
• stylelint 9.8.0
• supertest 3.3.0
• ts-jest 23.10.4
• ts-node 7.0.1
• tslint 5.11.0
• tslint-config-prettier 1.16.0
• tslint-eslint-rules 5.4.0
• tslint-microsoft-contrib 5.2.1
• tslint-react 3.6.0
• typescript 3.1.6

travis

.travis.yml

---

• Check this box to trigger a request for Renovate to run again on this repository

Actually use TypeScript types

Probably a good place to start? http://www.typescriptlang.org/docs/handbook/basic-types.html

Add some more tests

Let's get that 100% coverage!

Consider styled components

They're an option.

Create a TSLint configuration

• TSLint
• tslint-react
• tslint-eslint-rules
  There are probably more too.

Dependencies.io troubleshooting

We're opening this issue to inform you about problems with your dependencies.io configuration or build failures.

We'll add a comment here when we see problems. You can close this issue when
you have things resolved -- we'll open it back up if we detect new problems.

Helpful links:

• dependencies.io/docs
• https://app.dependencies.io/

Note: To help keep your repo as clean as possible, this is the only issue
that we're going to open and comment on. Feel free to delete the comments
that are made as you resolve them, if you want to keep this issue short and tidy.

Update sources' dependencies

Use renovate

Dependency deprecation warning: glamorous (npm)

On registry https://registry.npmjs.org/, the "latest" version (v5.0.0) of dependency glamorous has the following deprecation notice:

💄 glamorous has been deprecated in favor of 👩‍🎤 emotion. Please see https://github.com/paypal/glamorous/issues/419 for more information

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Affected package file(s): package.json

If you don't care about this, you can close this issue and not be warned about glamorous's deprecation again. If you would like to completely disable all future deprecation warnings then add the following to your config:

"suppressNotifications": ["deprecationWarningIssues"]

Add Scholarships to sources.dailybruin.com [Node, Express, React, GraphQL]

Dependency deprecation warning: babel-preset-es2015 (npm)

On registry https://registry.npmjs.org/, the "latest" version (v6.24.1) of dependency babel-preset-es2015 has the following deprecation notice:

🙌 Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!

Marking the latest version of an npm package as deprecated results in the entire package being considered deprecated, so contact the package author you think this is a mistake.

Affected package file(s): package.json

If you don't care about this, you can close this issue and not be warned about babel-preset-es2015's deprecation again. If you would like to completely disable all future deprecation warnings then add the following to your config:

"suppressNotifications": ["deprecationWarningIssues"]

Initial Login Hang

If the user logs in for the first time, the login page loads forever. If you then access the site again, it logs the user in.

Update sources table on mutation

Probably the last thing to do before sources is usable.

Add student orgs to sources

Requester: Marilyn Chavez (Outreach director 2020-2021)

Here is the format and data

https://docs.google.com/spreadsheets/d/1zw8x5nPSmbFahpEAjHxpli0gar3xudukQ1Wf26SzW5Q/edit?usp=sharing_eip&ts=6081ff39&urp=gmail_link&gxids=7628

We think the best way to add this is to make a separate tab for sources and for student orgs. In the sheet, each club has a finite number of categories. In the sources application, we want there to be no limit on the number of categories a club can have.

We also want the search bar to be able to search by category.

Fix Sources Build Errors [NPM, Typescript]

Temporary Solution

First, you will need to install node version 8.12.0.

Next run npm ci. THis will install directly from the package-lock.json instead computing the dependencies and then installing.

Finally, run npm start