cortezaproject / corteza

Low-code platform

Freedom. Flexibility. Performance. Speed.

The Corteza low-code platform lets you build and iterate CRM, business process and other structured data apps fast, create intelligent business process workflows and connect with almost any data source. All with the flexibility, freedom and control you get from a 100% open-source platform written on modern technologies maintained by Planet Crust, its founder.

Key Features

• Standards oriented by using well established data formats, technologies, and design decisions.
• Flexible security provided as a flattened RBAC facility allows organizations to apply complex internal security policies to Corteza.
• Extensive privacy features provide a way for organizations to configure Corteza to conform to data privacy regulations in the regions they are operating in.
• Accessible conforming to the WCAG 2.1 guidelines allowing simple use for a wider audience.
• Integratable into and with external services and other Corteza instances due to it's API-centric design, flexible data storage layer, and powerful automation facilities.
• Customizable from translations and terminology to prebuilt low-code applications to completely new low-code applications and business processes.

Getting Started

Note Planet Crust offers hosted Corteza cloud instances as the easiest way to get started.

Refer to the DevOps guide for a complete guide on how to get Corteza up and running. Additionally, we've provided some video instructions.

Quick references:

• data backup and restore
• troubleshooting
• setting up an email relay
• setting up sink routes
• offline and online configuration examples
• online demo instance for a hands-on feature exploration

Creating With Corteza

Refer to the Integrator Guide to learn how you can build on the core features to create virtually anything.

Quick references:

• Corteza Compose configuration
• automation using workflows and automation scripts
• using the REST API,

Upgrading Corteza

Refer to the changelog and the upgrade guide to upgrade your Corteza instance.

Contributing

Refer to the Developer Guide/Corteza Server document for details regarding the development setup, the project structure, and the feature insight documents.

Refer to the CONTRIBUTING.md for contribution guidelines and code of conduct.

Community

Reach out to us on our forum.

License

Corteza is released under the Apache-2.0 license. Refer to the LICENSE file for additional information.

corteza's Issues

Provide a way to get/set/update user roles

Provide a way to directly fetch/set/update the roles of a specific user.
Currently all role management is done through roles, so setting a role for a specific user is a bit of a pain.

session-store-secret leaks into logs

This project looks fantastic and useful for an internal dashboard of several applications and teams. Thanks in advance as we're looking to build off of Corteza!

Describe the bug
Some log lines contain sensitive information like:

system_1     | {"level":"info","ts":1566502103.9435697,"logger":"system.service.settings.discovery","caller":"service/settings_autodiscovery.go:93","msg":"value auto-discovered","name":"auth.external.session-store-secret","type":"string","default":"Brrtelyakaqtx8pGF4ONcCD5r4BdrTP4s5ct2gK8hCXsa2WGNCSlmSHc6eRJ1jB7"}

To Reproduce
Run the system docker image (cortezaproject/corteza-server-system:latest)

Expected behavior
Sensitive bits are masked so as to not appear in logs. I notice this is done elsewhere already.

Corteza Docker - unable to start the containers

I tried to install corteza docker containers according to the official HowTo guide.
https://docs.cortezaproject.org/corteza-docs/2020.6/maint/index.html#install

I have my docker up and running on Ubuntu 18.04.5 LTS with other cointainers happily running.

The problem is with corteza_correador cointainer - I see it as unhealthy.

This is in its log:

{"level":30,"time":1597409924933,"name":"check","msg":"server-scripts service configured","v":1}
{"level":30,"time":1597409924946,"name":"check","output":"/tmp/corredor/bundler-dist","msg":"making bundler output directory","v":1}
{"level":30,"time":1597409924956,"name":"services.dependencies","msg":"initializing watcher","v":1}
{"level":30,"time":1597409925027,"name":"services.server-scripts","searchPaths":["/extensions/*","/corredor/usr/*","/corredor/usr"],"msg":"reloading server scripts","v":1}
{"level":30,"time":1597409925300,"name":"services.client-scripts","searchPaths":["/extensions/*","/corredor/usr/*","/corredor/usr"],"msg":"reloading client scripts","v":1}
{"level":30,"time":1597409929236,"name":"services.client-scripts","valid":19,"total":19,"msg":"processed","v":1}
{"level":30,"time":1597409936971,"name":"services.server-scripts","valid":14,"total":14,"msg":"processed","v":1}
{"level":30,"time":1597409937070,"name":"gRPC","msg":"server running at corredor:8082","v":1}

Also corteza_server is stopped.

{"level":"warn","ts":1597409907.093877,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":1,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409912.1021376,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":2,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409917.10366,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":3,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409922.1083221,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":4,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409927.11075,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":5,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409932.1142328,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":6,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409937.116536,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":7,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409942.1180427,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":8,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409947.1191387,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":9,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409952.1213322,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":10,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409957.1226094,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":11,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}
{"level":"warn","ts":1597409962.1240015,"logger":"database","caller":"db/connector.go:66","msg":"could not connect to the database","name":"default","error":"dial tcp 172.20.0.2:3306: connect: connection refused","errorVerbose":"dial tcp 172.20.0.2:3306: connect: connection refused\ngithub.com/titpetric/factory.(*DatabaseFactory).Get\n\t/drone/src/vendor/github.com/titpetric/factory/database.go:142\ngithub.com/cortezaproject/corteza-server/pkg/db.TryToConnect.func1\n\t/drone/src/pkg/db/connector.go:64\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1373","try":12,"dsn":"c****a:c****e@tcp(db:3306)/corteza?collation=utf8mb4_general_ci","delay":5}

Error: could not connect to database: db init for "default" timedout

I appreciate any help how to resolve this.

Thank you

Messaging email notifications

Add ability to send messages from messaging service to users when user is mentioned or new message is added to channel or thread.

User should be able to control this setting:

• disable notifications
• instant notifications (on each event but within a certain time-window)
• digest (hourly, daily)

Settings are global and per-channel (inherit global as default)

Users should be able to mute notifications via email link.

Allow record sorting by reference field

Syntax proposal:
<ref-field-1>.<ref-field-2>.<some-field>

Extract permissions / access-control logic to external package

Duplicate entry error when trying to save a record with a multiple select

Describe the bug
I have a module with a record field that allows multiple select. I can select them, but when I save, I get: "Could not update this record: exec query failed. Error 1062. Duplicate entry '112231180576751620-MailingList-0' for key 'PRIMARY'"

To Reproduce
Steps to reproduce the behavior:

1. Go to https://latest.cortezaproject.org/compose/ns/service-cloud/pages/100056965736038402/record/112231180576751620/edit
2. In the "mailing list" field on the right, select multiple users
3. Hit save
   -> the error "Could not update this record: exec query failed. Error 1062. Duplicate entry '112231180576751620-MailingList-0' for key 'PRIMARY'" shows

Expected behavior
I expect to be able to save multiple records.

Screenshots

Desktop (please complete the following information):

• Firefox (latest)

Additional context
If I select only 1 record in the "mailing list" field, it saves without problem. This error only shows with 2 or more selected values.

Reload cached settings on change

When setting values are altered via API or CLI - reload cache.

Remove PHP tooling for converting API spec to controllers, request handlers etc..

GitHub login does not include required email

Describe the bug
"can not use profile data without an email"
from https://api.latest.cortezaproject.org/system/auth/external/github/callback
Is produced when trying the latest.cortezaproject.org installation with a GitHub account.

To Reproduce
Make sure you're logged out and proceed to https://latest.cortezaproject.org/auth/login
Then try to use a github account for logging in.

Expected behavior
Login should succeed.

Screenshots
n/a

Additional context
You may be requesting the wrong OAuth scope for the GitHub API.
The code seems to request email, however this document mentions user:email.

Going to my account at https://github.com/settings/applications it shows corteza has gotten "Access public information (read-only)", as opposed to some other apps which include "Access user email addresses (read-only)"

Do not require email confirmation on existing accounts when email confirmation is disabled

Are there any PBX/CSTA/TAPI/webRTC integration hooks or APIs yet?

Or is there anything planned?

Design process, flow & API for data removal requests (GDPR)

PostgreSQL support

Hello,

I would like to have the possibility to use PostgreSQL because my entire information system is only with PostgreSQL.

I'm looking to use OpenERP/Odoo/Tryton because they use PostgreSQL but in same time it's very time and money consuming to adapt them to be similar as Corteza.

Thank you

Improve webapp package building

• All frontend apps (admin, unify, compose, messaging, auth) and common should (?) be built and pushed to package repository (npm, yarn or github)
• common package should be prebuilt and included (instead of doing gitrepo ref)
• verify if corteza webapp package could be prebuilt and included into crust

Translations

Is your feature request related to a problem? Please describe.
I think this project would be a good fit for some people that I know, but sadly they're not fluent in English.

Describe the solution you'd like
It would be great if this project supported translations, so that the community could then translate the software to their own languages.

Describe alternatives you've considered
I've looked through the docs and the code, but couldn't find support for translations. Without technical support, I can't think of an alternative.

Can't start in kubernetes with corredor enabled

Describe the bug
I've deployed corteza in my kubernetes cluster, but to get it running I had to set CORREDOR_ENABLED=false. That's because if I don't do that, I get this error and the server fails to start:

Error: CA public key (/certs/corredor/client/ca.crt) not found 

To Reproduce
I've attached a kubernetes yaml file that (given the mysql database exists, which I provision elsewhere with kubedb) will deploy the service in a way that works. If you remove CORREDOR_ENABLED=false it fails to boot with the above-listed error.

Expected behavior
I expected the server to boot or to be able to read in the documentation something related to a certificate required to talk to corredor. I also just expected corredor to boot with the rest of the system, but I have only learned corteza existed yesterday so am well aware that my mental model is likely lacking.

Additional context
I can boot the app and use messaging / administer users / use the jitsi bridge if i disable corredor, but crm and servicecloud don't work, and compose doesn't really work though it will list namespaces and let me create one. I assume these use corredor heavily but again, no real clue.

Here's a kubernetes setup that describes what I'm trying to do at least...

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: corteza-server
  namespace: corteza
  labels:
    app: corteza-server
  annotations: {}
spec:
  strategy:
    rollingUpdate:
      maxUnavailable: 0
    type: RollingUpdate
  selector:
    matchLabels:
      app: corteza-server
  template:
    metadata:
      labels:
        app: corteza-server
    spec:
      containers:
        - name: corteza-server
          image: cortezaproject/corteza-server:2020.3.0-rc.2
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
          env:
            - name: CORREDOR_ENABLED
              value: "false"
            - name: LOG_LEVEL
              value: debug
            - name: LOG_DEBUG
              value: "true"
            - name: VIRTUAL_HOST
              value: api.corteza.dbadbadba.com
            - name: HTTP_CLIENT_TSL_INSECURE
              value: "true"
            - name: PROVISION_SETTINGS_AUTH_FRONTEND_URL_BASE
              value: https://corteza.dbadbadba.com
            - name: MYSQL_USERNAME
              valueFrom:
                secretKeyRef:
                  name: corteza-mysql-auth
                  key: username
            - name: MYSQL_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: corteza-mysql-auth
                  key: password
            - name: DB_DSN
              value: "$(MYSQL_USERNAME):$(MYSQL_PASSWORD)@tcp(corteza-mysql:3306)/corteza?collation=utf8mb4_general_ci"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: corteza-webapp
  namespace: corteza
  labels:
    app: corteza-webapp
  annotations: {}
spec:
  strategy:
    rollingUpdate:
      maxUnavailable: 0
    type: RollingUpdate
  selector:
    matchLabels:
      app: corteza-webapp
  template:
    metadata:
      labels:
        app: corteza-webapp
    spec:
      containers:
        - name: corteza-webapp
          image: cortezaproject/corteza-webapp:latest
          imagePullPolicy: IfNotPresent
          ports:
            - name: http-webapp
              containerPort: 80
              protocol: TCP
          env:
            - name: API_BASEURL
              value: api.corteza.dbadbadba.com
            - name: VIRTUAL_HOST
              value: corteza.dbadbadba.com
            - name: MONOLITH_API
              value: "true"
          resources:
            limits:
              cpu: 1000m
              memory: 512Mi
            requests:
              cpu: 100m
              memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
  name: corteza-server
  namespace: corteza
  labels:
    app: corteza-server
spec:
  ports:
    - port: 80
      protocol: TCP
  selector:
    app: corteza-server
---
apiVersion: v1
kind: Service
metadata:
  name: corteza-webapp
  namespace: corteza
  labels:
    app: corteza-webapp
spec:
  ports:
    - port: 80
      protocol: TCP
  selector:
    app: corteza-webapp
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: corteza-server
  namespace: corteza
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.org/websocket-services: "corteza-server"
spec:
  tls:
    - hosts:
        - api.corteza.dbadbadba.com
      secretName: corteza-server-tls
  rules:
    - host: api.corteza.dbadbadba.com
      http:
        paths:
          - path: /
            backend:
              serviceName: corteza-server
              servicePort: 80
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: corteza-webapp
  namespace: corteza
  annotations:
    kubernetes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
    - hosts:
        - corteza.dbadbadba.com
      secretName: corteza-webapp-tls
  rules:
    - host: corteza.dbadbadba.com
      http:
        paths:
          - path: /
            backend:
              serviceName: corteza-webapp
              servicePort: 80

Revert 7beecc50efb0dde7589667b8164a1b9c3f31b9b8

Implements an one-time-tool for id correction. Should be removed as it doesn't need to exist in the future.

Support for additional external record storage (ERS) connection adapter

In addition to native connection (using existing connection to corteza db), add support for additional record storages.

This adapter needs to support all CRUD & filtering operations to fetch & store records from and to external storage.

Examples:

• use records tables inside primary database (current & default behaviour)
  • no extra configuration is needed
  • 2 tables are used record + record_values
• use a dedicated physical table to store records inside primary database
  • admin can set a dedicated db table
  • ??? how to solve multi-value fields
  • ??? should table exist or will we create it?
• interface to MongoDB
• interface to ElasticSearch
• interface to a specific endpoint on external REST API
• …

New ERS are enabled and configured through general admin interface.

All interaction with records should be then transparently handled by adapter.

Module configuration interface is extended - admin can then pick one of the adaptors and provide extra config if needed

Finalise implementation of privacy/masking settings

Currently, we ignore privacy.mask.email and privacy.mask.name settings.

Remove fixed values from system/service/user.go and use values from settings.

sonyflake causing crash on startup

docker run -e DB_DSN="corteza:change-me@(250.2.89.8:3306)/corteza" cortezaproject/corteza-server-system

{"level":"info","ts":1566740547.1042073,"logger":"system.database","caller":"db/connector.go:77","msg":"connected to the database","name":"system","dsn":"ca:ce@(250.2.89.8:3306)/corteza"}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x84f4b7]

goroutine 1 [running]:
github.com/cortezaproject/corteza-server/vendor/github.com/sony/sonyflake.(*Sonyflake).NextID(0x0, 0x0, 0x0, 0x0)
/go/src/github.com/cortezaproject/corteza-server/vendor/github.com/sony/sonyflake/sonyflake.go:89 +0x37
github.com/cortezaproject/corteza-server/vendor/github.com/titpetric/factory.(*SonyflakeFactory).NextID(...)
/go/src/github.com/cortezaproject/corteza-server/vendor/github.com/titpetric/factory/sonyflake.go:31
github.com/cortezaproject/corteza-server/system/internal/repository.(*application).Create(0xc0001d7520, 0xc000187320, 0x20, 0xb60780, 0x20)
/go/src/github.com/cortezaproject/corteza-server/system/internal/repository/application.go:71 +0x35
github.com/cortezaproject/corteza-server/system.makeDefaultApplications.func1(0xc000187320, 0xc45a40, 0x13cf201)
/go/src/github.com/cortezaproject/corteza-server/system/autosetup.go:68 +0x12a
github.com/cortezaproject/corteza-server/system/types.ApplicationSet.Walk(0xc00014b830, 0x1, 0x1, 0xc00014b848, 0x0, 0x0)
/go/src/github.com/cortezaproject/corteza-server/system/types/application.gen.go:18 +0x60
github.com/cortezaproject/corteza-server/system.makeDefaultApplications(0xdb8f80, 0xc00018c030, 0xc00020c000, 0xc000001680, 0x0, 0x0)
/go/src/github.com/cortezaproject/corteza-server/system/autosetup.go:60 +0x34c
github.com/cortezaproject/corteza-server/system.Configure.func3(0xdb8f80, 0xc00018c030, 0xc00020c000, 0xc000001680, 0xc000188390, 0xc13ce0)
/go/src/github.com/cortezaproject/corteza-server/system/system.go:60 +0x67a
github.com/cortezaproject/corteza-server/pkg/cli.Runners.Run(0xc00000e638, 0x1, 0x1, 0xdb8f80, 0xc00018c030, 0xc00020c000, 0xc000001680, 0xc00014bcc8, 0x8)
/go/src/github.com/cortezaproject/corteza-server/pkg/cli/runner.go:116 +0x7f
github.com/cortezaproject/corteza-server/pkg/cli.(*Config).MakeCLI.func2(0xdb8f80, 0xc00018c030, 0x0, 0x0)
/go/src/github.com/cortezaproject/corteza-server/pkg/cli/runner.go:262 +0xaa
github.com/cortezaproject/corteza-server/pkg/api.(*Server).Command.func1(0xc00020c280, 0x13cf218, 0x0, 0x0, 0x0, 0x0)
/go/src/github.com/cortezaproject/corteza-server/pkg/api/server.go:65 +0x1bb
github.com/cortezaproject/corteza-server/vendor/github.com/spf13/cobra.(*Command).execute(0xc00020c280, 0xc00000c090, 0x0, 0x0, 0xc00020c280, 0xc00000c090)
/go/src/github.com/cortezaproject/corteza-server/vendor/github.com/spf13/cobra/command.go:751 +0x519
github.com/cortezaproject/corteza-server/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc00020c000, 0xdb8f80, 0xc00018c030, 0xc00020c000)
/go/src/github.com/cortezaproject/corteza-server/vendor/github.com/spf13/cobra/command.go:852 +0x2ec
github.com/cortezaproject/corteza-server/vendor/github.com/spf13/cobra.(*Command).Execute(...)
/go/src/github.com/cortezaproject/corteza-server/vendor/github.com/spf13/cobra/command.go:800
main.main()
/go/src/github.com/cortezaproject/corteza-server/cmd/system/main.go:11 +0x9e

Add exporter endpoints

Add simple (….yaml) endpoints on modules, records, … that outputs entire structure as (importable) YAML

Allow record filtering by reference field

Syntax proposal:
ref-field-1>.<ref-field-2>.<some-field> = "foo"

Masked usernames are shown for (fake) system users

On provision, server creates a couple of channels, we need to be able to disable automatic message creation

@#### ######## created public channel General

Suppress these kind of messages on server

Add ability to import and export applications YAML

Move email templates to the database (settings)

• Move templates from system/internal/service/auth_notification.go to db (settings) to allow per-instance customization.
• setting needs to be provisioned on first-run (or in case of missing entry, same as other auth settings)
• implement setting of values (rest/settings.go) via API (see #5)

/cc @lenny-horstink

uploads error: Cannot read property 'progress' of undefined

Describe the bug
I was trying to upload a few CSV files and when they error'd I was kicked out of the upload dialog to a javascript error: Cannot read property 'progress' of undefined

To Reproduce
Upload either of these files:

Non-ASCII character?

FirstName,LastName,Email,Company
P.g,ផល្លី),[email protected],pp

Improper amount of commas

FirstName,LastName,Email,Company
R.n,H.r,,S.a, LLC.,Github/ach

Expected behavior
The uploads should either fail within the dialog or skip, but I don't think bailing out entirely is the correct move.

Screenshots

Additional context
When uploading I had "Cancel Import" selected, but after the error I'm forced to refresh the entire Corteza tab.

Make OIDC provider registration async and resilient

OIDC provider registration should be done asynchronously and allow registration
re-try. When registration is complete, reinitialize goth. It might be smart to reinit
goth after every OIDC provider registration (when there is more than 1).

Potential issues:

1. goth lib does not play nice with re-initialization of providers

Enforce field types on record values

Add backend logic for record value type enforcement on input and output.

Server routine crashes on (some?) websocket disconnects

[20:58:32][COR] : 2019-12-10T20:58:32.819+0100        ERROR        websocket/websocket.go:61        websocket session handler error        {"error": "writeLoop send: websocket: close sent", "errorVerbose": "websocket: close sent\nwriteLoop send\ngithub.com/cortezaproject/corteza-server/messaging/websocket.(*Session).writeLoop\n\t/Users/darh/Work.crust/corteza-server/messaging/websocket/session.go:255\ngithub.com/cortezaproject/corteza-server/messaging/websocket.(*Session).Handle\n\t/Users/darh/Work.crust/corteza-server/messaging/websocket/session.go:166\ngithub.com/cortezaproject/corteza-server/messaging/websocket.Websocket.Open\n\t/Users/darh/Work.crust/corteza-server/messaging/websocket/websocket.go:60\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/go-chi/chi.(*Mux).routeHTTP\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:424\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/cortezaproject/corteza-server/messaging/websocket.Websocket.ApiServerRoutes.func1.1.1.1\n\t/Users/darh/Work.crust/corteza-server/messaging/websocket/router.go:59\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/go-chi/chi.(*Mux).ServeHTTP\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:69\ngithub.com/go-chi/chi.(*Mux).Mount.func1\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:291\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/go-chi/chi.(*ChainHandler).ServeHTTP\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/chain.go:31\ngithub.com/go-chi/chi.(*Mux).routeHTTP\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:424\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/go-chi/chi.(*Mux).ServeHTTP\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:69\ngithub.com/go-chi/chi.(*Mux).Mount.func1\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:291\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/cortezaproject/corteza-server/pkg/auth.(*token).HttpAuthenticator.func1.1\n\t/Users/darh/Work.crust/corteza-server/pkg/auth/jwt.go:136\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/go-chi/jwtauth.Verify.func1.1\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]/jwtauth.go:82\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/go-chi/chi.(*ChainHandler).ServeHTTP\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/chain.go:31\ngithub.com/go-chi/chi.(*Mux).routeHTTP\n\t/Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:424\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/766b/chi-prometheus.Middleware.handler.func1\n\t/Users/darh/Work/go/pkg/mod/github.com/766b/[email protected]/middleware.go:60\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/getsentry/sentry-go/http.(*Handler).Handle.func1\n\t/Users/darh/Work/go/pkg/mod/github.com/getsentry/[email protected]/http/sentryhttp.go:59\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/cortezaproject/corteza-server/pkg/api.HandlePanic.func1\n\t/Users/darh/Work.crust/corteza-server/pkg/api/middleware.go:57\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995\ngithub.com/cortezaproject/corteza-server/pkg/api.LogResponse.func1\n\t/Users/darh/Work.crust/corteza-server/pkg/api/logger.go:75\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1995"}
[20:58:32][COR] : github.com/cortezaproject/corteza-server/messaging/websocket.Websocket.Open
[20:58:32][COR] :         /Users/darh/Work.crust/corteza-server/messaging/websocket/websocket.go:61
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).routeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:424
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/cortezaproject/corteza-server/messaging/websocket.Websocket.ApiServerRoutes.func1.1.1.1
[20:58:32][COR] :         /Users/darh/Work.crust/corteza-server/messaging/websocket/router.go:59
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).ServeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:69
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).Mount.func1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:291
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi.(*ChainHandler).ServeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/chain.go:31
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).routeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:424
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).ServeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:69
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).Mount.func1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:291
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/cortezaproject/corteza-server/pkg/auth.(*token).HttpAuthenticator.func1.1
[20:58:32][COR] :         /Users/darh/Work.crust/corteza-server/pkg/auth/jwt.go:136
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/jwtauth.Verify.func1.1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]/jwtauth.go:82
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi.(*ChainHandler).ServeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/chain.go:31
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).routeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:424
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/766b/chi-prometheus.Middleware.handler.func1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/766b/[email protected]/middleware.go:60
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/getsentry/sentry-go/http.(*Handler).Handle.func1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/getsentry/[email protected]/http/sentryhttp.go:59
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/cortezaproject/corteza-server/pkg/api.HandlePanic.func1
[20:58:32][COR] :         /Users/darh/Work.crust/corteza-server/pkg/api/middleware.go:57
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/cortezaproject/corteza-server/pkg/api.LogResponse.func1
[20:58:32][COR] :         /Users/darh/Work.crust/corteza-server/pkg/api/logger.go:75
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/cortezaproject/corteza-server/pkg/api.LogRequest.func1
[20:58:32][COR] :         /Users/darh/Work.crust/corteza-server/pkg/api/logger.go:52
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/cortezaproject/corteza-server/pkg/api.contextLogger.func1.1
[20:58:32][COR] :         /Users/darh/Work.crust/corteza-server/pkg/api/logger.go:30
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi/middleware.RequestID.func1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/middleware/request_id.go:72
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi/middleware.RealIP.func1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/middleware/realip.go:34
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/cors.(*Cors).Handler.func1
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]/cors.go:199
[20:58:32][COR] : net/http.HandlerFunc.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1995
[20:58:32][COR] : github.com/go-chi/chi.(*Mux).ServeHTTP
[20:58:32][COR] :         /Users/darh/Work/go/pkg/mod/github.com/go-chi/[email protected]+incompatible/mux.go:81
[20:58:32][COR] : net/http.serverHandler.ServeHTTP
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:2774
[20:58:32][COR] : net/http.(*conn).serve
[20:58:32][COR] :         /usr/local/go/src/net/http/server.go:1878
[20:58:32][COR] : 2019-12-10T20:58:32.821+0100        INFO        http.rest        api/logger.go:65        HTTP response GET /messaging/websocket/        {"requestID": "Deniss-MacBook-Pro.local/mlQfQgIVwH-000632", "method": "GET", "path": "/messaging/websocket/", "status": 0, "size": 0, "duration": 761.519860895}
[21:01:41][COR] : 2019-12-10T21:01:41.586+0100        DEBUG        api/monitor.go:69        monitor        {"alloc": 3879144, "totalAlloc": 121845608, "sys": 73335032, "mallocs": 1956270, "frees": 1921835, "liveObjects": 34435, "pauseTotalNs": 34977739, "numGC": 102, "numGoRoutines": 57}
^C[21:05:08][COR] : 2019-12-10T21:05:08.831+0100        INFO        http        api/server.go:161        HTTP server stopped
[21:05:08][COR] : 2019-12-10T21:05:08.831+0100        DEBUG        system.grpc-server        system/system.go:101        shutting down        {"addr": "localhost:50052"}

Healthcheck endpoints & docker config

Diagnostics / checks

Provide mechanism to run diagnostics on all configuration settings, support and auxiliary services. Should display notices, warnings, errors and suggestions on how to solve certain problems.

These checks should be runnable from CLI and (later) via API. Add ability to cache results if cache can easily invalidated (config value).

Checks that need to be covered in the first phase:

• Corredor
  • is service reachable,
  • can scripts be executed
  • can scripts reach back to the API
• SMTP
  • can email be sent (send email to the auth FROM address)
• Authentication
  • can users log in (according to settings)
  • can users register (according to settings)
  • what kind of 3rd party auth mechanisms are enabled

SMTP_FROM possibly ignored

Describe the bug
Using the Docker simple instructions a Corteza server instance with SMTP env variables set and carried into the image do not result in working mailing. When signing up I get

Error: could not send email authentication notification: gomail: could not send email 1: gomail: invalid address "": mail: no address

To Reproduce
download example .env file.
set SMTP settings
use docker-compose up -d
sign up for new account
Expected behavior
A new account email should be sent.

Additional context
SMTP variables appear to be present in the server environment when I check inside of them.
Not sure if SMTP_FROM is being ignored or if when signing up the mailto is not being read properly.

Migrations need to specify a primary key on table creation

Describe the bug
Unable to initialize a fresh DB when sql_require_primary_key is set in MySQL

To Reproduce
Steps to reproduce the behavior:
Create a fresh DB, set sql_require_primary_key and run the stack, allowing migrations to run.

Expected behavior
Expected the migration scripts to be compatible with this MySQL best practice.

Additional context
Currently using DigitalOcean for managed MySQL services:

MySQL databases containing tables without a primary key and which contain more than 5000 rows may experience replication issues. To prevent this, DigitalOcean now requires you to add a primary key for each new table you create in any managed MySQL database created after 8 April 2020. We strongly recommend that you also add primary keys in existing databases to avoid replication issues.

I would be happy to open a PR with adjusted migration scripts if you can link the source files and statik asset bundling script.

Improve CLI UX

Disable verbose log output that we currently have and allow user to enable it via CLI flags

Upgrade to go 1.13

Improve ENV variable handling

All *Opt structs (pkg/cli/options) could be generated from (YAML) configuration. In addition, we could generate documentation - how can options be set via ENV variables and what is the effect

Migrate to v1.13 errors

Errors should be unified and moved under types package

Refactor pkg/count

pkg/count is too fragmented.
I would suggest we join it with compose/decoder (only one using it)

https://blog.golang.org/package-names

@tjerman thoughts?

Request Index abilities for columns

To accelerate the search, or to support the unique column feature, it's a request to build indexes on the compose_record_values.

A possible solution may be to use the indexes pivot table like force.com.

https://developer.salesforce.com/page/Multi_Tenant_Architecture
http://www.developerforce.com/media/ForcedotcomBookLibrary/Force.com_Multitenancy_WP_101508.pdf

Add ability to import and export users as YAML

Cleanup orphaned files

See #3

• messaging attachments
  • check how/when attachments are deleted (after message is deleted)
• compose attachments
• add backend process (see service.Watchers()) to cleanup orphaned files after XY hours/days (should be configurable)

Remove current Web-hook implementation

The current implementation of webhooks is partly supporting the need for
inbound webhooks. This implementation was never finished nor it has any
UI. With the introduction of /sink endpoint (done) and ability to write and run (system)
automation scripts (pending) there will no need for it anymore.

At this moment, webhooks (as implemented) are NOT used anywhere and
should be removed to avoid confusion and lower complexity.

Remove all Webhook related code

• Repository
• Service
• Rest endpoints & specs
• Types
• Tests

Add support for unique values

Module field should accept 2 types of unique constraint - per module (single- or multi-value fields) and per record (multi-value fields).

Provide an ability to provision/import custom (group) of YAML files via CLI.

Validate record values

Add backend logic for record value validation, compatible with frontend validators.

Setup performance testing (k6, …?)

Add ability to import and export records as YAML

Connectivity errors for internal gRPC

While provisioning monolith, messaging & compose subsystems (this is an assumption!) are waiting for system to initialise it’s gRPC server.

Can this be fixed with order of service initialization?
Can we do this without gRPC (thin(er) srv-clt) in monolith env?
Should clients be more patient/less verbose on first connection?

server_1    | {"level":"info","ts":1576043812.908581,"logger":"compose","caller":"service/service.go:176","msg":"initializing corredor connection","addr":"corredor:80"}
server_1    | {"level":"info","ts":1576043812.9086642,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043812.9087434,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc0002b1630, CONNECTING"}
server_1    | {"level":"info","ts":1576043812.9089036,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043812.9089322,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043812.9096236,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc0002b1630, READY"}
server_1    | {"level":"info","ts":1576043813.9091003,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043813.9091728,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043813.909203,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043815.6602032,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043815.66034,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043815.660389,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043818.3077023,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043818.3079214,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043818.3079727,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043823.078922,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043823.0790393,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043823.0790734,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043830.5522795,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043830.552378,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043830.5524154,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043842.9193661,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043842.919565,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043842.9196188,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043862.1911037,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043862.191241,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043862.1912994,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}
server_1    | {"level":"info","ts":1576043887.5598469,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, CONNECTING"}
server_1    | {"level":"info","ts":1576043887.5601141,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"grpc: addrConn.createTransport failed to connect to {:50051 0  <nil>}. Err :connection error: desc = \"transport: Error while dialing dial tcp :50051: connect: connection refused\". Reconnecting..."}
server_1    | {"level":"info","ts":1576043887.5602112,"logger":"compose.service.grpc","caller":"zapgrpc/zapgrpc.go:94","msg":"pickfirstBalancer: HandleSubConnStateChange: 0xc000378f20, TRANSIENT_FAILURE"}

Allow upload to module fields w/o existing record

Currently, we expect a record to exist before we can do a file upload.
Users should be able to upload files directly to modules and bind that to record later

Todo:

• new upload endpoint /namespace/{namespaceID}/module/{moduleID}/attachment/{fieldName}
• alter existign upload endpoint for consistencty: /namespace/{namespaceID}/module/{moduleID}/record/{recordID}/attachment/{fieldName}
• alter AttachmentService.CreateRecordAttachment to accept calls w/ recordID=0 and to verify permissions (CanCreateRecord)
• add backend process (see service.Watchers()) to cleanup orphaned files after XY hours/days (moved to #4)

See:

• cortezaproject/corteza-webapp-compose#79