openstad / openstad-api

OpenStad API server

openstad-api

Setup

1. Installatie

[email protected]:Amsterdam/openstad-api.git
cd openstad-api
npm i

2. Maak een lokale configuratie file ./config/local.js

module.exports = {

  "url": "URL",
  "hostname": "DOMAIN",

  "database": {
    "user": "DB_USERNAME",
    "password": "DB_PASSWORD",
    "database": "DB_NAME",
    "host": "DB_HOST",
    "dialect": "DB_DIALECT",
    "multipleStatements": true
  },

  "express": {
    "port": PORT
  },

  "mail": {
    "from": "EMAIL_ADDRESS",
    "transport": {
      "smtp": {
        "port": SMTP_PORT,
        "host": "SMTP_HOST",
        "auth": {
          "user": "SMTP_USERNAME",
          "pass": "SMTP_PASSWORD"
        }
      }
    }
  },

  "security": {
  },

  "authorization": {
    "jwt-secret": "MIJNOPENSTAD_JWT_SECRET",
    "auth-server-url": "MIJNOPENSTAD_URL",
    "auth-client-id": "MIJNOPENSTAD_DEFAULT_CLIENT_ID",
    "auth-client-secret": "MIJNOPENSTAD__CLIENT_PASSWORD",
    "auth-server-login-path": "/dialog/authorize?redirect_uri=[[redirectUrl]]&response_type=code&client_id=[[clientId]]&scope=offline",
    "auth-server-exchange-code-path": "/oauth/token",
    "auth-server-get-user-path": "/api/userinfo?client_id=[[clientId]]",
    "auth-server-logout-path": "/logout?clientId=[[clientId]]",
    "after-login-redirect-uri": "/?jwt=[[jwt]]",
    "fixed-auth-tokens": [
      {
        "token": "12345",
        "userId": "2"
      }
    ]
  }
}

3. Initeëer de database

For a successful reset run, make sure the following ENV values are properly set:

#unique hash
FRONTEND_URL=
AUTH_URL=

#unique hash
AUTH_FIRST_CLIENT_SECRET=
AUTH_FIRST_CLIENT_ID=

#for instance = http://localhost:4444|https://admin.openstad.demo
ADMIN_URL=
#optional
AUTH_INTERNAL_SERVER_URL=

Then run following command:

env ... node reset.js

If you are using an existing database and only want to migrate run the following command:

node -r dotenv/config migrate.js

4. Start de server

node server.js

5. Run tests

There are several test scripts:

1. npm run test For all tests scenarios
2. npm run test:unit For all unit tests
3. npm run test:e2e For all e2e tests

Travis CI

GITOPS

Gitops steps:

• Test application
• Generate assets
• Build docker image
• Push docker image
• Commit new image to helm repo (based on GITOPS_RELEASE_BRANCH and GITOPS_{environment}_VALUES_FILE )

If you want to enable gitops flow in the ci pipeline of travis you need to configure a few variables:

• GITOPS=true
• DOCKER_PUBLIC_USERNAME=openstad
• DOCKER_IMAGE_NAME=api
• HELM_REPO_NAME=openstad-kubernetes
• HELM_CHART_FOLDER=k8s/openstad
• GITOPS_RELEASE_BRANCH=master
• GITOPS_DEV_VALUES_FILE=k8s/openstad/environments/dev.values.yaml
• GITOPS_ACC_VALUES_FILE=k8s/openstad/environments/acc.values.yaml
• GITOPS_PROD_VALUES_FILE=k8s/openstad/environments/prod.values.yaml

MySQL with SSL

When you want to connect to a MySQL server using SSL, a Certificate Authority certificate is required. The contents of this CA certificate can be passed into the MYSQL_CA_CERT environment variable.

Documentatie

Meer informatie staat in de docs directory.

openstad-api's Issues

Use one mail transporter for all sites

The mail routine needs a site config to determine which transporter to use. Since it seems highly unlikely that different mail transports would be used for different sites this should be changed. One mail transporter for the whole application should suffice.

When this is done it would be possible to merge messages to the same email address into one email, even when hey target different sites, in e.g. src/cron/send_site_issues_notifications.js

Sanitize util transform null into the string 'null'

Whenever the sanitize function (src/util/sanitize.js) is called with null, it returns the string 'null'.

When does this matter?
When adding an oauth client which does not have lastName as a required field, the lastName will be set tonull upon creating an account and going through the required fields page. After logging in, the API creates a user from these values. The User model has lastName set to allow null values, but the setter is always passing the value through to the sanitize util. This means that when you create a user called 'Admin' in the oAuth environment, the user will be called 'Admin null' in the API environment. This can be seen in the resource-form-widget's form header for example.

Merge notification systems

The code contains two notification services.

The first can be found in ./src/services/notificationService.js which is used by ./src/services/eventService.js. This uses rulesets, and seems to be an attempt towards a generic and flexible notification system. The problem is that it is only used by Submissions.

The second can be found in ./src/notifications.js. It uses queues which are processed by crons. This one is used by the rest of the API.

These services should be merged.

Add generic cron locks

Since PR #254 a locking mechanism is available.

The next step should be to create a generic use of this mechanism by cron tabs.

module.exports = {
	cronTime: '0 2 */6 * * *',
	runOnInit: false,
	onTick: createLockForCron(async function() {
	  // cron logic
	})
};

Optimization: move email templates from `site.config` to it's own field

It would be nice to move email templates from the site config to a new field.

site.config is a json field with a lot of data. All this data is requested from the database again and again.

The email templates are currently stored in site.config. This means that these templates are also requested from the database again and again. But this data is hardly ever used.

Moving these templates to another field in the site table, and only request those fields when required, would significantly reduce ttraffic between the API and database server, and between the API and the frontend server.