GithubHelp home page GithubHelp logo

orbiting / backends Goto Github PK

View Code? Open in Web Editor NEW
39.0 2.0 11.0 15.94 MB

[DEPRECATED] moved to https://github.com/republik/plattform monorepo for republik, publikator and assets backends based on NodeJS providing the GraphQL API used to power republik.ch.

Home Page: https://api.republik.ch/graphiql

License: GNU Affero General Public License v3.0

JavaScript 79.93% PLpgSQL 1.84% HTML 13.79% Shell 0.50% HCL 0.01% TypeScript 3.92% Procfile 0.01%
nodejs graphql apollo postgresql redis republik docker backend kibana elasticsearch

backends's Introduction

[DEPRECATED] @orbiting/backends Build Status Coverage Status

⚠️ This repository together with others has been merged into the republik/plattform monorepo. Let's continue the journey there. ⚠️

This repo contains all the backend code in use at Republik. For easier development the previously separate repos republik-backend, publikator-backend, assets-backend and backend-modules where merged into this monorepo.

For a guide on how to start the frontends see: docs/how-to-run

Components

The components in this repo are split into two subfolders:

  • servers: contains runnable servers
  • packages: contains code shared between the servers

How to run / Development

1. Clone

git clone [email protected]:orbiting/backends.git && cd backends

2. Prerequisites

You must have Node.js (10+), yarn, docker and docker-compose installed (alternatively to docker you can install the external services natively).

Docker

The included docker-compose.yml starts all external-services. Currently that's: postgresql, redis, elasticsearch (and kibana). The data is persisted in ./docker-data/.

docker-compose up [-d]
Postgresql in docker

We recommend you install the postgresql client tools on your machine to interact with the database. The tests scripts also depend on the clients being installed.

# linux
sudo apt install postgresql-client-12

When postgresql in running in docker client tools like psql or createdb/dropdb don't automatically connect to it. They try to access postgresql via a local socket, when instead you want them to connect via network to localhost. To make your life easier, you can add the following environment variables to ~/.bashrc / ~/.zshrc so the client tools connect to localhost per default.

export PGHOST=127.0.0.1
export PGUSER=postgres

alternativ: install natively

show more

As an alternative to docker(-compose) you can install the external-services natively:

On macOS with homebrew:

brew install postgresql redis nvm elasticsearch
nvm install 14
nvm alias default 14
npm install -g [email protected]
brew services start postgresql
brew services start redis
brew services start elasticsearch

Docker Kibana accessing native Elasticsearch

docker run -p 5601:5601 -e ELASTICSEARCH_HOSTS=http://host.docker.internal:9200 docker.elastic.co/kibana/kibana-oss:6.7.0

Note:

  • Elasticsearch and Kibana versions must match, ckeck ES version at http://localhost:9200/
  • ELASTICSEARCH_HOSTS must be accessible within docker.

2. ENVs

Copy the .env.example files to .env (in root and servers/assets/). The default values should be enough to get started.

cp .env.example .env
cp servers/assets/.env.example servers/assets/.env

3. Install

yarn install

4. Setup

yarn dev:setup

5. Run

yarn dev

This kicks on foreman which then launches all the servers locally. All servers greets you with Cannot GET / on the root route. The API server has a graphical API explorer available at /graphiql:

Next steps

more about ENVs

In development environment variables are loaded from ./.env. No ENV variabeles are loaded from any file in production, you yourself are responsible to set all required ENVs in the production environment.

Checkout .env.example, servers/assets/.env.example for which ENVs are required and their descriptions. Check the packages' README for further config options.

You will quickly run into errors and limitations if you run with the example envs. You probably want to do the following two rather soon:

  1. MailChimp and Mandrill
    • MAILCHIMP_URL, MAILCHIMP_API_KEY, MANDRILL_API_KEY in .env
    • MAILCHIMP_* in .env (less important)
  2. S3 Bucket
    • AWS_* in the root .env

Special setup: develop on two hosts

The following setup enables to start the servers (backends and republik-frontend) on one machine (A) and access it from another (B). This can come handy if you want to develop the backend on A and the app on B (where B can be a physical device).

Please not that due to how "Docker for Mac" works (docker is run in a hidden VM), it's not possible to bind containers to the host's network-interface, therefore this setup only works on linux.

Machine A (servers)

  1. Get the IP of your machine in the local network, use it in the next step as LOCAL_IP
ip addr
  1. Adapt hostnames in the environment variables:
  • in backends/.env
FRONTEND_BASE_URL=http://republik.test
ASSETS_SERVER_BASE_URL=http://assets.republik.test

LOCAL_IP=192.168.1.88

CORS_ALLOWLIST_URL=http://republik.test
COOKIE_DOMAIN=.republik.test
  • in republik-frontend/.env
API_URL=http://api.republik.test/graphql
API_WS_URL=ws://api.republik.test/graphql
API_ASSETS_BASE_URL=http://republik.test
PUBLIC_BASE_URL=http://republik.test
  1. Start the DNS-Server and reverse proxy:
docker-compose -f docker-compose-test-network.yml up [-d]
  • bind: You now have a DNS server running locally. It resolves all requests of *.republik.test to LOCAL_IP.
  • traefik: routes requests based on SNI (check: traefik.toml)
    • http://republik.test -> http://localhost:3010
    • http://api.republik.test -> http://localhost:5010
  1. Run backend services with docker (in backends/):
docker-compose up [-d]
  1. Refresh published articles, due to changed ASSETS_SERVER_BASE_URL (in backends/):
yarn pull:elasticsearch
redis-cli
> FLUSHALL
  1. Run the backend servers (in backends/):
yarn dev
  1. Run the frontend server (in republik-frontend/):
npm run dev
  1. Test You should be able to access http://api.republik.test/graphiql and http://republik.test

Machine B (app)

  1. To resolve the hostnames:
  • Find the IP of Machine A, let's say its 192.168.1.88
  • change your network config to use this IP as your DNS resolver.
  1. Adapt hostnames in the environment variables (in app/.env.dev):
API_URL=http://api.republik.test/graphql
API_WS_URL=ws://api.republik.test/graphql
FRONTEND_BASE_URL=http://republik.test
  1. Test

You should be able to access http://api.republik.test/graphiql and http://republik.test

  1. Setup simulators/emulators

At least the android emulator doesn't use the hosts dns resolver

  • configure the DNS resolver manually (see step 1) inside the simulator
  • test in the webbrowser that http://republik.test is accessible.
  • run the app as usual (in app/)
yarn run-android

Postfinance Import

see packages/republik-crowdfundings/lib/scheduler/payments/README.md.

Licensing

The source code and it's documentation is licensed under GNU AGPLv3+.

backends's People

Contributors

annatraussnig avatar caillou avatar clarajeanne avatar dependabot[bot] avatar eightam avatar fuenkchen avatar garamond avatar lukasbuenger avatar ovbm avatar patrickvenetz avatar patte avatar pozylon avatar ruggedly avatar salim-b avatar tpreusse avatar trm217 avatar uxengine avatar werehamster avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

backends's Issues

If membership result comes from cache, applyPgInterval throws an error

Bug Report

If membership is provided via cache, gracePeriod is not a PostgresInterval object. applyPgInterval.mutate however expects a PostgresInterval object.

Steps to Reproduce

  1. Query for User.activeMembership.graceEndDate
  2. Delete (Redis) cache key crowdfundings:cache:User:<User ID>:membership:<Membershp ID>:graceEndDate
  3. Run query again

Actual Results

Error is thrown:

graphql error in undefined (undefined): { Error: interval missing (should be "PostgresInterval")
    at mutate (/app/packages/utils/applyPgInterval.js:19:11)
    at add (/app/packages/utils/applyPgInterval.js:40:28)
    at createMembershipCache.cache (/app/servers/republik/modules/crowdfundings/graphql/resolvers/Membership.js:96:16)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  message: 'interval missing (should be "PostgresInterval")',
  locations: [ { line: 1, column: 314 } ],
  path:
   [ 'users', 'items', 197, 'activeMembership', 'graceEndDate' ] }

Expected Results

A date

Refactor Slack Messages API

The Slack messages API is currently inconsistent.

// packages/slack/lib/publish.js
publish()
postMessage()

The publish() and postMessages() mehtods in https://github.com/orbiting/backends/blob/master/packages/slack/lib/publish.js do not return promises, nor do any of the methods in https://github.com/orbiting/backends/blob/master/packages/republik/lib/slack.js.

Yet, most of the code relies on these methods returning promises.

There is an attempt at refactoring the code here: 4ef98c2

Yet, the questions remains open, if these methods should return promises at all.

It is event questionable, if the Slack API should be used in our code, or if we should just log to the console and react to it separately.

Fix mergeUsers mutation with regard to answers and other constraints

As of now, the userId of an answer can not be modified due to the following PG trigger:

IF (
TG_OP = 'UPDATE' AND
NEW."userId" IS NOT NULL AND
NEW."userId" != OLD."userId"
) THEN
RAISE EXCEPTION 'PSQL EXCEPTION: answers must not switch users';
END IF;

A possible workarround could be this: 0ad0aca

As of now, I am not certain if this PG trigger is important, and if the above code change would break something.

Additionally, mergeUsers() might fail due to unic constraints. We need to figure out a good way to automatically merge such users.

GraphQL error: Cannot set property 'topValue' of undefined

Required article redirects are missing

Bug Report

It won't create a redirect under some circumstances which causes 404 errors in newsletters.

Steps to Reproduce

  1. Publish Article v1 on date
  2. Schedule a revised Article v2 to be publish on a later date
  3. Create a newsletter and push it to MailChimp. Link in newsletter points to v1.
  4. Wait until Article v2 is published

Actual Results

There is no redirect installed from v1 to v2. Once newsletter is sent, v1 does not exist anymore and renders 404 error.

Expected Results

There is a redirect setup to point from v1 to v2. Once newsletter is sent, v1 redirect to v2.

Possible Solutions

Setup redirect if URL changes from one version to the other.

Remove commented out code

In the following commit, some code has been commented out, as the code was not fully tested and somewhat critical:

eac5e53

Once we are sure this method is called as expected, we should un-comment the code.

switch monthly -> yearly results in unsubscribe newsletters

when a user decided to switch from a monthly to a yearly membership he/she get's unsubscribed from newsletters because all memberships are inactive for a short moment.
possible solution: don't force unsubscribe if activatable memberships are present

Discussion path changes when discussion document is only scheduled

Bug Report

URL of an already published discussion is updated when only scheduling (discussion) document.

Steps to Reproduce

  1. Create a discussion document
  2. Publish discussion document
  3. Change publication date
  4. Schedule publication of discussion document

Actual Results

discussions.path is updated upon scheduling.

Expected Results

discussions.path should only be updated while publishing.

Upgrade to ABO_GIVE_MONTHS generates year long period

Bug Report

When "upgrading" from MONTHLY_ABO to ABO_GIVE_MONTHS, webhook from Stripe upon subscription cancellation will create a year long period instead of initial interval/period set in ABO_GIVE_MONTHS membership.

Steps to Reproduce

  1. Create MONTHLY_ABO
  2. Let other user buy ABO_GIVE_MONTHS
  3. Claim ABO_GIVE_MONTHS
  4. Cancel MONTHLY_ABO (immediately).

Other

As of July 27, 2020 no actual cases to be found in database.

Several problems while running the application

Bug Report

Steps to Reproduce

I installed a web application according to how-to-run guide. Successfully added a new user. But when I go to the profile, I see the following:

Actual Results

screen-shot

Also the log contains a stack-trace:
republik.log

Environment

  • Node 10.15.3/yarn 1.9.4
  • PostgreSQL 11.2-3
  • Redis 4.0.9
  • Elasticsearch 6.7.0
  • OS: Arch Linux (Kernel 5.0.10)

Please, can you check it out and tell what is running wrong in my case. Thank you.

Jest test accidentally loaded on Travis

Bug Report

Booting the backend on travis fails:
https://travis-ci.com/orbiting/republik-tests/jobs/195254711#L3125

/home/travis/build/orbiting/republik-tests/apps/backends/packages/utils/ensureStringLength.jest.js:3
describe('ensureStringLength', () => {
^
ReferenceError: describe is not defined
    at Object.<anonymous> (/home/travis/build/orbiting/republik-tests/apps/backends/packages/utils/ensureStringLength.jest.js:3:1)
    at Module._compile (internal/modules/cjs/loader.js:701:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
    at Module.load (internal/modules/cjs/loader.js:600:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
    at Function.Module._load (internal/modules/cjs/loader.js:531:3)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at proxy (/home/travis/build/orbiting/republik-tests/apps/backends/node_modules/lazy-cache/index.js:26:21)
    at module.exports (/home/travis/build/orbiting/republik-tests/apps/backends/node_modules/export-files/index.js:29:39)
    at Object.<anonymous> (/home/travis/build/orbiting/republik-tests/apps/backends/packages/utils/index.js:1:103)
    at Module._compile (internal/modules/cjs/loader.js:701:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
    at Module.load (internal/modules/cjs/loader.js:600:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
    at Function.Module._load (internal/modules/cjs/loader.js:531:3)

This happens because

  1. export-files is used for packages/utils which contains a jest file ensureStringLength.jest.js adding a lazy ensureStringLengthJest export to the utils package
  2. export-files uses lazy-cache and lazy-cache has a explicit condition not to be lazy on travis
    // check both boolean and string in case `process.env` cases to string
    if (process.env.UNLAZY === 'true' || process.env.UNLAZY === true || process.env.TRAVIS) {
      cache[name] = fn(mod);
    }

Steps to Reproduce

  • TRAVIS=1 yarn run dev

Actual Results

crash with ReferenceError: describe is not defined

Expected Results

run without an issue

Possible Solutions

  • get ride of export-files, jest files should not even be available behind a lazy getter

Workaround

Set travis env to an empty string on travis:

TRAVIS= PORT=5000 SERVER=republik yarn start 

Test 3rd 🌊

  • PROLONG pledge
    • membership_owner_prolong_notice_0 (T-0)
    • membership_owner_prolong_notice_7 (T-7)
    • membership_owner_prolong_notice (T-30)
    • membership_deactivated_* (T+14)
  • AutoPay
    • membership_owner_autopay_notice (T-10)
  • changeover, from membership A to membership B
  • /account view
  • Prolong notification in header
  • Admin-Tool

Fixes:

  • /account view: Do not display cancel link if dormant membership is used next

Feature Request: Automatic Import of Postfinance Data

Automatic Import of Postfinance Data

Current Situation

As of now, the inport if Postfinance Data is a semi-manual task. These are the steps:

  • Log in to the E-Finance web platform.
  • Download CSV with payment information.
  • Download PDF containing images of orange payment slips.
  • Log in to Orbiting admin web platform.
  • Upload the CSV to the admin web platform.
  • Manually match payments that do not contain a HR-ID, either using a user information or the payment slip image.
  • Manually send payment reminders.
  • Manually deactivate users that are way overdue.

Feature Request

This feature request aims to reduce the time spent and possiblity of errors when importing payment data as follows:

  • Automatically request payment information from Postfinance.
  • Automatically match payment information.
  • If there are unmatched payments:
    • Inform admin users about unmatched payments (Slack, email, …).
    • Offer a UI to the admin users to efficiently match payments.
  • Automatcially send out payment reminders.
  • Automatically deactivate users that are way overdue.

Technical Information

Postfinance FDS / EBICS

Postfinace offers the following options to automatically retrieve payment information:

Pro Contra
EBICS
  • International standard.
  • Changing bank should be simple.
  • New-ish standard in Switzerland.
  • Complex implementation, node library is fairly recent and untested.
  • More overhead with Postfinance to get an account.
FDS
  • Very simple implementation.
  • Very well tested node library.
  • Less overhead with Postfinance to activate connection.
  • Propiretary Postfinance standard.

Due to the lesser administrative overhead with Postfinance and the well-tested node library, it would be avisable to take the route of FDS.

ISO 20020

Both, FDS and EBICS, will return the payment information following an ISO 20020 standard, namely camt.053 with suplementay orang payment slipt images.

This implies that the current code that imports a CSV needs to be adapted to import a camt.053.xml file.

Postfinance Test Environment

While Postfinance offers a free testing environment, it differs for their production environment insofar as files are delivered as ZIP files, rather than plaintext. It might not actually make a lot of sense to use their environment for development.

Tasks

  • Rewrite import to accept camt.053.xml files.
  • Notify admin users of unmatched payments.
  • Automatically send out overdue notifications upon matching.
  • Automatically deactivate users who are way overdue.
  • Set up a development server for testing and development.
  • Connect and download the files.
  • Convert and save payment slip images.
  • Store jpeg files.
  • GrapQL Endpoints:
    • List of unmatched payments with images.
    • Matching mutation.
  • Imlement a matching wizard on the admin web platform:
    • Display images for orange payment slips.
    • Offer an inline search for Name and/or Address in order to easily match for the information on the orange payment slip or information in the reference message.
    • The list should idally contain the following fields for matching: pledge amount, name, adderss

Open Questions

Storing Orange Payment Slip Images

It might make sense to store the orange payment slip images as base64 encoded blobs in the DB. These are very small, can be deleted upon matching a payment, and could be returned as base64 images in the GraphQL responses.

support markdown in comments

API change (servers/republik/graphql/schema-types.js):

type Comment {
  ...
  # maybe becomes mdast/JSON later
  content: String
  ...
}

becomes:

type Comment {
  ...
  # mdAST
  content: JSON
  # content as string
  text: String
  ...
}

submitComment stays the same.

parse supported types:

ABO_GIVE_MONTHS prolonging flow incomplete

Bug Report

When ABO_GIVE_MONTHS is prolonged (regular 1 year), some business logic is not working or not defined yet:

  • no prolong notice transactional emails are sent. As of July 30, 2020, that affects ~50 memberships
  • if auto payment is enabled, system won't put this membership into charge cycle. As of July 30, 2020, that affects ~15 memberships.

publikator: Links with tel URI rendered faulty

Bug Report

Adding a link using tel URI is rendered will render link faulty.

Steps to Reproduce

  1. Create an article in Publikator with a link pointing to tel:+41 34 56 78.
  2. Publish article.

Actual Results

Link href is filled with tel:+41203420562078.

It appears it replaces white spaces with %20 and then strips %.

Expected Results

Either tel:+41%2034%2056%2078 (or tel:+41345678).

Include video durations in estimatedConsumptionMinutes

Bug Report

Steps to Reproduce

{
  document(path: "/2019/06/08/das-verlorene-paradies") {
    meta {
      totalMediaMinutes
      estimatedReadingMinutes
      estimatedConsumptionMinutes
    }
    content
  }
}

Live Query

Additional issue: if content is not requested totalMediaMinutes is 0 because it's resolved at request time.

Actual Results

{
  "data": {
    "document": {
      "meta": {
        "totalMediaMinutes": 19,
        "estimatedReadingMinutes": 4,
        "estimatedConsumptionMinutes": 4
      },
      "content": {} // clipped
  }
}

Expected Results

{
  "data": {
    "document": {
      "meta": {
        "totalMediaMinutes": 19,
        "estimatedReadingMinutes": 4,
        "estimatedConsumptionMinutes": 23
      },
      "content": {} // clipped
  }
}

Possible Solutions

  • preprocess totalMediaMinutes at index time
  • add totalMediaMinutes to estimatedConsumptionMinutes unless meta.audioSource.durationMs is present
    • this should still win for e.g. «An der Bar» cases where there is a video and audio file of the same thing and same duration

Comment.preview shortens faulty

Bug Report

Comment.preview is shortening phrases wrong.

mdastToHumanString splits phrase into parts and then stitches parts together until desired length is reached.

In an attempt to reach desired length it skips parts while stitching which leads to odd preview texts.

Pseudo example, shortening to 95 chars this quote:

You know the greatest danger facing us is ourselves, and irrational fear of the unknown. There is no such thing as the unknown. Only things temporarily hidden, temporarily not understood.

Actual Results

You know the greatest danger facing us is ourselves, and irrational fear of the unknown. is...

It skipped "There" but following word "is" still fit into desired length.

Expected Results

You know the greatest danger facing us is ourselves, and irrational fear of the unknown...

Possible Solutions

Fix horribly bad coding. Might be caused by tree structure. So, split into parts first, then stitch them together.

`check-env` invisible when deploying

port from: orbiting/backend-modules#2

The check env exception are hard to notice on Heroku.

It would be nice to add a preinstall (or probably heroku-prebuild to avoid local dev annoyance) script that performances the check. This would also avoid an deploy going though and then bringing down an app because of an missing env var.

Trouble installing the backend on linux

Hey there,

(do you prefer english or german? I am german, but github issues, seem to prefer english)

thanks for developing your stack open source. I am basically a noob, trying to get this running on my laptop in order to learn. I do this, since the fridays for future people, in particular the netto.null group seemed interested... again not an expert here at all.

So I followed the how-to-run in the docs and started by trying to setup the backend README.

(installing yarn on ubuntu 18.04 was a bit of a hassle due to cmdtest ... but I think I got it running after all)

I am stuck at point 4 init yarn run db:migrate:up returned this error msg:

(base) ✔ ~/climate_strike/netto.null_website/backends [master|✔] 
10:05 $ yarn run db:migrate:up
yarn run v1.16.0
error @orbiting/[email protected]: The engine "node" is incompatible with this module. Expected version "12.x". Got "10.16.0"
error Commands cannot run with an incompatible environment.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

(base) ✘-1 ~/climate_strike/netto.null_website/backends [master|✔] 
10:05 $ node --version
v10.16.0

So indeed my node version is 10.16 , while something somewhere expects 12.x ... I do not understand where I messed this up? Is there a more recent version of node running inside one of the docker containers?

Do you have any idea how to proceed? Should I just try to update my host node to 12.x and start over from scratch?

Thanks a lot for your great work.

Membership Renewal Notification Flow

Membership Renewal Notification Flow

(DRAFT)

A membership can enter a renewal notification flow.

It currently split into three stages.

stage (A)         -> stage (B)              -> stage (C)
initial reminder  -> 7-days-before-reminder -> 2-days-before-reminder

A scheduler will query memberships. It will place memberships in according stage (depending on a set of rules). When placing a membership in a particular stage. It may act by sending an email or notification to Slack.

This scheduler won't tamper wont any membership or user data. Deactivating is job of other listener, or schedulers.

Stage of a membership can be queried via GraphQL.

{
  membership {
    prolong {
      state
    }
  }
}

Flow Stages

Stage is stored in membership, and it's transitions in flowStages

entity date flow stage source ...
123-123-123 2019-12-15 renewal stage (A) scheduler
123-123-123 2019-01-08 renewal stage (B) ...
123-123-123 2019-01-13 renewal stage (C)
123-123-123 ...

flowStages contains other flows e.g. for on-boarding purposes, preview mailings.

Scheduler has a series of "stage" checks, run sequentially:

Renewal Flow

Stage Checks

x) query: last periods ending T-52
- "Kampagne"
(- manual, one-off activity)
* set stage (A)

x) query: last periods ending T-2
- "Kampagne"
(- manual, one-off activity)
* set stage (A)

x) query: last periods ending T-45
- "Geschenk"
(- is membership active? [implicit via query])
- is membership pledger, but not claimer?
- is pledge-membership-combo not in promotion queue?
* send email
* set stage (A)

x) query: last periods ending T-45
- "Gönner"
- is package BENEFACTOR?
(- is membership active? [implicit via query])
- is membership pledger and claimer?
- is membership in prolong queue?
- is pledge-membership-combo not in promotion queue?
* notify via Slack
* set stage (B)

x) query: last periods ending T-30
- "Normalfall"
(- is membership active? [implicit via query])
- is membership pledger and claimer?
- is pledge-membership-combo not in promotion queue?
* send email
* set stage (A)

x) query: last periods ending T-7
- "Normalfall"
(- is membership active? [implicit via query])
- is membership pledger and claimer?
- is membership.renew set to false
- is pledge-membership-combo in stage-a, nor not in queue?
* send email
* set stage (B)

x) query: last periods ending T-2
- "Normalfall"
(- is membership active? [implicit via query])
- is membership pledger and claimer?
- is membership.renew set to false
- is pledge-membership-combo in stage-b, nor not in queue?
* send email
* set stage (C)

Buggy Mandrill HTML to Text conversation

      Sie erhalten diese E-Mail, weil Sie an der Debatte =C2=ABD=C3=BCrfen =
wir mehr
=C3=BCber Ihr Verhalten wissen?
<https://www.republik.ch/dialog?id=3D57e2e394-f269-4244-9678-6a62a6cce99a&a=
mp;t=3Darticle>=C2=BB
teilgenommen haben und/oder Benachrichtigungen f=C3=BCr diese abonniert hab=
en. 

Should be:

      Sie erhalten diese E-Mail, weil Sie an der Debatte =C2=ABD=C3=BCrfen =
wir mehr
=C3=BCber Ihr Verhalten wissen?
<https://www.republik.ch/dialog?id=3D57e2e394-f269-4244-9678-6a62a6cce99a&t=
=3Darticle>=C2=BB
teilgenommen haben und/oder Benachrichtigungen f=C3=BCr diese abonniert hab=
en. 

Mandrills converter fails to convert &amp; to & when extracting hrefs from the html version. In HTML attributes like href & should be encoded as &amp; but in plain text obviously not. This is a bug in Mandrills auto_text converter.

Possible solution:

  • write our own converter and send precomputed plain text version
  • maintain all emails as plain text too

Possible workaround:

  • skip &amp; encoding in hrefs, most likely most html parser could handle it. But this would not be spec conform ({{{URL}}} instead of {{URL}} in handlebar templates).

Cannot read property 'indexOf' of undefined ("url")

Bug Report

Error log reports following message now and then in republik-assets-api server:

(node:15091) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'indexOf' of undefined
    at whitelistedUrls.find.whiteUrl (/app/packages/assets/express/render.js:20:46)
    at Array.find (<anonymous>)
    at /app/packages/assets/express/render.js:20:25
    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
    at next (/app/node_modules/express/lib/router/route.js:137:13)
    at Route.dispatch (/app/node_modules/express/lib/router/route.js:112:3)
    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
    at /app/node_modules/express/lib/router/index.js:281:22
    at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
    at next (/app/node_modules/express/lib/router/index.js:275:10)
    at /app/node_modules/regiment/lib/middleware.js:39:7
    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/app/node_modules/express/lib/router/index.js:317:13)
    at /app/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
    at next (/app/node_modules/express/lib/router/index.js:275:10)

May be request in which url is not set in req.query.

Feature: employees endpoint

Follow-up to #66 as discussed with @patte.

We can't use the search API directly on a public page like /impressum

So we need some public API endpoint like employees(userIds: []): [User!]!. You also suggested to combine that query with the gsheet data. (This will affect frontend code which currently retrieves the ghseet data in a first step, see orbiting/republik-frontend#136)

Sign-in-Link does not work with certain Web-mail-clients

Bug Report

I have recently changed my email address for my Republik-Account to one using Open Exchange Web Frontend. There, the Sign-in-Link is encoded wrong and is invalid:
https://www.republik.ch/mitteilung?type=token-authorization&email=mail.address%**2540**provider.ch&context=signIn&token=TOKEN

The issue lies with the double encoded @. I am uncertain, if this could be an issue with OX Web or the link, as it used to work on Gmail.

Steps to Reproduce

  1. Sign out
  2. Login with Email
  3. Click link in OX Web mail client

Actual Results
https://www.republik.ch/mitteilung?type=token-authorization&email=mail.address%2540provider.ch&context=signIn&token=TOKEN

Expected Results
https://www.republik.ch/mitteilung?type=token-authorization&email=mail.address%40provider.ch&context=signIn&token=TOKEN

Environment

  • OS: Fedora 27
  • Firefox 59.0.2 (64-bit)
  • UI version: 7.8.4 Rev28
  • Server version: 7.8.4-Rev29

Possible Solutions

Monatsabo: E-Mail-Erinnerung "Kreditkarte erneuern" trotz Kündigung

Bug Report

Nachdem die Kreditkarte abgelaufen ist und der Kunde deswegen das Monatsabo kündigt (beispielsweise, um auf ein Jahresabo zu wechseln), erhält er weitere Mails, die ihn dazu auffordern, die Kreditkarte zu erneuern.

Steps to Reproduce

Leserin besitzt ein Monatsabo. Nun läuft ihre Kreditkarte ab.
Das Monatsabo hat sich bereits verlängert. Leserin erhält nun Mails, die sie auffordern, die Kreditkarte zu erneuern.
Leserin entscheidet nun, das Monatsabo zu kündigen (optional: und auf ein Jahresabo zu wechseln.)

Actual Results
Die Aufforderungen per E-Mail erhält sie nun nach wie vor, was meistens damit endet, dass das Monatsabo auf die Dauer (nachdem 1 Woche lang die Kreditkarte nicht erneuert wurde), sowieso storniert wird.

Expected Results
Wenn ich ein Monatsabo besitze, dessen Kreditkarte abgelaufen ist, würde ich erwarten, dass die Kündigung ein 'sofort canceln' auslöst, und ich keine weiteren E-Mails mehr erhalte.
Optional: Wenn ich auf ein Jahresabo umsteige, erwarte ich, dass mein (abgelaufenes) Monatsabo sofort storniert wird und mein Jahresabo aktiv wird. Somit erhalte ich ebenfalls keine weiteren E-Mails, die mich darauf aufmerksam machen, dass meine Kreditkarte abgelaufen ist.

Environment
Unabhängig von der verwendeten Umgebung.
(Kreditkarte erfassen sowieso nicht via App möglich)

Possible Solutions
(optional auto generated) Kündigung eines Monatsabo mit abgelaufener Kreditkarte bewirkt im System 'sofort canceln' des Monatsabos.

pullElasticsearch iterates github repos twice in parallel resulting in abortion due to github's ratelimit

When running yarn run pull:elasticsearch all repos are requested twice. This can lead to abortion due to the ratelimit of github (message: You have triggered an abuse detection mechanism. Please wait a few minutes before you try again.)

The two repo iterations happen here:

The later was introduced to index repos for publikator search:
https://github.com/orbiting/backends/pull/98/files#diff-e36345a8c8f9e51ebdcf08e813d51de8

Temporary workaround: Pull each index individually:

yarn run pull:elasticsearch --index document
yarn run pull:elasticsearch --index repo

`updateUser` buggy

Bug Report

Requesting email in an updateUser mutation explodes:

{
  "data": {
    "updateUser": {
      "email": null
    }
  },
  "errors": [
    {
      "locations": [
        {
          "column": 108,
          "line": 1
        }
      ],
      "message": "Cannot read property 'emailAccessRole' of undefined",
      "path": [
        "updateUser",
        "email"
      ]
    }
  ]
}

After mutating successfully. Possibly a transform user missing?

Steps to Reproduce

mutation {
  updateUser(userId: "ID", firstName: "Barbara", lastName: "Muster") {
    email
  }
}

Actual Results

Error

Expected Results

return email.

Possible Solutions

  • properly transform user

Improve RSS support

I read all my news inside my RSS reader (I use Newsify on iOS but the proposed solution should work with any RSS reader). As such I currently rely on services without a paywall (though I happily pay for The Guardian and Taz).

So as a paying Republik customer I would like to have a custom RSS feed URL, that would contain pre-authenticated links to articles. This means if I open the article on my RSS reader, the URL would “log me in” to Republik and so the RSS reader would be able to open the article, so I can read directly in my RSS reader.

Note technically the pre-authenticated URL should just do the login and then redirect to a non pre-authenticated URL, so that when I share the URL of the article, I would not leak the pre-authenticated URL but the normal public URL.

Potentially in that case optionally the RSS feed should already contain the entire content of the articles, to make it easier to read offline and to just include the public URL as the reference, to further reduce the chance of leaking the pre-authenticated URL.

I would also be fine if the pre-authenticated URL would only work 2-3 times to again prevent the risk of people free-loading off of accidentally pre-authenticated URLs.

Of course the token added to the URLs to handle the pre-authentication should be specific to each article.

If this feature would be provided, I would happily subscribe again to Republik.

Forward HTTP Status Code for GitHub in Asset Server

Bug Report

E.g. 404 get cached in the cdn because the status code is not forwarded:

https://cdn.repub.ch/github/republik/article-wdwww-01-05/images/f859740c39115c2bafcd4929da5bf65f6a29e59c.jpeg?size=2864x1909
is a 404 but asset backend returns a 200.

Somehow a bad credentials request – probably a 400 – was cached as a 200 and prevented a image from showing up correctly in publikator even after a while. Directly requesting it worked (assets instead of cdn). If the error would have been properly handled it would have self healed. Now I had to manually clean the cdn cache for that url.

Comment.preview is not shortening URLs

Bug Report

Comments displayed on /dialog ("Feed") using Comment.preview prop is not shortening URLs. This heightens the likelyhood to break out of a container on narrow screens such as mobile devices.

URL shortening

It does so when rendering a comment in its entire glory on a discussion page.

Steps to Reproduce

  1. Add a comment with an URL without dashes ("-"). URL should appear early on to be visible above the preview fold of 500 chars.

Verlängerung Abo_Give_Months

Bug Report

Wenn ich mehrere Monate verschenkt habe, habe ich unter dem Tab 'Verlängerung' nur die Möglichkeit, für die Person mit 240 CHF zu verlängern.
Zudem taucht diese Verlängerungsoption an einem 'seltsamen' Ort auf - nämlich oberhalb meiner eigenen Verlängerung und nicht unterhalb, wie die anderen Geschenkeinkäufe, die ich bestellt habe.

Steps to Reproduce

-Monatsabo als Geschenk für jemanden kaufen
-Auf die Verlängerungsseite gehen
-Dort sieht man, dass die Möglichkeit fehlt, weitere Monate zu verschenken

Actual Results

image

Expected Results

Das Verlängern taucht unterhalb meines eigenen Abonnements auf. Zudem habe ich die Möglichkeit, erneut auszuwählen, um wie viele Monate ich verlängern möchte.

Ensure userid in mailLog

Most logs in the mailLog table are insterted without a userId.

Ensure that the userId is always set and use the id in order to request the logs.

Normalize mailLog table

As of now, the mailLog table identifies a user by their e-mail address.

This leads to errors, when users change their addresses.

Refactor the mailLog to user the users' id rather than the e-mail address.

Differenciate Anonymous Comments

When reading comments, we sould be able to differenciate annonymous comments written by the same users.

If we look at the following example, both Ada Lovelace and Grace Hopper wrote anonymous comments. Taken that there are 4 anonymous, it is not currently possible to know, how many users took part in the discussion.

Screenshot 2020-08-27 at 18 31 31

Broken Stats without PARKING_PLEDGE_ID/PARKING_USER_ID

Running the backend with empty/not set PARKING_PLEDGE_ID and PARKING_USER_ID fails on memberStats queries.

Steps to Reproduce

Actual Results

query memberStats {
  memberStats {
    count
  }
}
graphql error in undefined (undefined): { error: invalid input syntax for type uuid: ""
10:40:56 AM republik.1  |      at Connection.parseE (/Users/diego/Desktop/OpenSource Projects/backends/node_modules/pg/lib/connection.js:546:11)
10:40:56 AM republik.1  |      at Connection.parseMessage (/Users/diego/Desktop/OpenSource Projects/backends/node_modules/pg/lib/connection.js:371:19)
10:40:56 AM republik.1  |      at Socket.<anonymous> (/Users/diego/Desktop/OpenSource Projects/backends/node_modules/pg/lib/connection.js:114:22)
10:40:56 AM republik.1  |      at Socket.emit (events.js:182:13)
10:40:56 AM republik.1  |      at Socket.EventEmitter.emit (domain.js:442:20)
10:40:56 AM republik.1  |      at addChunk (_stream_readable.js:283:12)
10:40:56 AM republik.1  |      at readableAddChunk (_stream_readable.js:264:11)
10:40:56 AM republik.1  |      at Socket.Readable.push (_stream_readable.js:219:10)
10:40:56 AM republik.1  |      at TCP.onread (net.js:635:20)

Expected Results

Do not send the empty id to DB. Run query without error.

Environment

  • Node/yarn version: Node 10.4.0/Yarn 1.7.0

Possible Solutions

Do not send parking ids if not set.

E-Mails: Text-Logik bei 'sofort canceln' eines Monatsabos

Bug Report

Wenn wir im Admintool ein Monatsabo 'sofort canceln' und die Kündigungsbestätigung nicht unterdrücken, wird eine E-Mail (Bestätigung Ihrer Kündigung) ausgelöst, die als Kündigungsdatum den letzten Tag des 'regulär weiterlaufenden' Monatsabos ausweist.

Steps to Reproduce
-Kundin hat zB eine Kreditkarte, die nicht mehr belastet werden kann
-Monatsabo hat sich bereits verlängert
-Kundin schreibt, sie will per sofort kündigen
-im Admintool klicken wir auf 'sofort canceln', E-Mail wird ausgelöst

Actual Results

Guten Tag [Name]
Sie haben sich entschieden, die Republik nicht weiter zu abonnieren. Das bedauern wir sehr.
Wir haben Ihre Kündigung erfasst: Ihr Abonnement läuft noch bis zum Datum in Zukunft und wird nicht erneuert.
Wir danken Ihnen herzlich für Ihr Interesse und für Ihre Unterstützung im vergangenen Jahr!Und wir würden uns sehr freuen, wenn Sie zu einem späteren Zeitpunkt wieder bei uns vorbeischauen.
Sie sind jederzeit willkommen.
Ihre Crew der Republik und von Project R
--

Expected Results

Datum Heute

Possible Solutions
'Sofort canceln' ändert das Datum in der Kündigungsbestätigung auf den heutigen Tag
alternativ:
'sofort canceln' löst eine andere E-Mail aus

Dringlichkeit:
Nicht so hoch, kann umgangen werden mit 'Kündigungsbestätigung unterdrücken' - da wir meistens sowieso in persönlichem Austausch mit Kundin stehen.

E-Mails: Text-Logik bei Verlängerung eines Geschenkabos ohne eigenes Abo

Bug Report

Reported by Jonas.

Betrifft Mail bei Erneuerung von Geschenkabos. Da steht (auch wenn man selbst kein Abo hat oder das eigene bereits verlängert wurde) zuerst:

[Vorname Name], willkommen im zweiten Jahr der Republik!

Es ist schön, dass Sie weiterhin dabei sind! Hier die wichtigsten Informationen über Ihre Verlängerung.

und dann:

Gratulation für Ihre Grossherzigkeit: Sie verlängern auch Ihre Geschenkabonnemente. Wir haben die Beschenkten darüber per E-Mail informiert.

Da müsste man wohl unterscheiden, was genau verlängert wurde, bzw. ob das eigene Abonnement auch dazugehört oder nicht.

Extends logic in backends/packages/mail/templates/pledge_prolong.html to cover cases when one did not prolong their own or create separate email.

Improve database query when prefix is used in adminUsers query

Enhancement

When using adminUsers GraphQL query with a recognized prefix (e.g. nr: or hrid:) adopt query request to database to increase speed and lower load. Could be achieved by removing unnecessary strings in searchable field and removing unnecessary table joins.

First attempts resulted in a speed-up of up to 50%.

Refers to a suggestion submitted by @patte.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.