enmasseproject / enmasse Goto Github PK

From @lulf on June 9, 2017 11:2

With the introduction of ready status on address through the address API, the tests should check this when deploying configuration.

Copied from original issue: EnMasseProject/systemtests#15

This makes it easier to deploy EnMasse on public cloud providers that do not have an ingress controller.

From @lulf on March 29, 2017 7:35

Add a garbage collection of unused instances (0 addresses defined) configurable when deploying the address controller.

Copied from original issue: EnMasseProject/admin#3

From @lulf on March 30, 2017 15:29

Making third party resources of v3.Instance and v3.Address allow instances and addresses to be created using openshift and kubernetes tooling (api server and authentication for free). Work required:

• Create ThirdPartyResource definitions for Instance and Address
• Create controllers in address-controller that watches for instance and address resources and performs actions (instantiate templates, deployments etc.)
• Authorize users to create/delete address resources in the namespace of their instance.
• Change the current HTTP API to create these resources rather than acting on the 'internal' objects directly. Creating this API instance can be optional.

Copied from original issue: EnMasseProject/admin#6

From @ppatierno on March 1, 2017 10:49

If I send a subscription request to the subserv with one valid topic and one NOT existing topic (i.e. mytopic exists, mytopyc not exist) can I assume that receiving this error :

description='Error: Unrecognised topic: mytopyc'

the subserv has rejected the whole request so the client isn't subscribed to the existing topic ?

Is the subserv behavior something like ... "all valid" -> ok , "at least one not valid" -> ko ?

Just to be sure, because in any case I can't distinguish between who is valid and who is not valid.

Copied from original issue: EnMasseProject/subserv#15

This would be a supplement to the deployment script, and simplify operations that manages their infrastructure using ansible. Hawkular is perhaps a good example to start from.

From @ppatierno on April 7, 2017 10:41

When a receiver connects on a topic, the subscribers chart appears but when the receiver disconnects, it doesn't disappear.

Copied from original issue: EnMasseProject/routilities#4

From @ppatierno on May 31, 2017 13:56

Instead of spinning up two different containers in the same mqtt pod for having non-TLS and TLS enabled mqtt server, we could have only one container and both mqtt server instance in the same mqtt gateway instance (i.e. see the Eclipse Hono approach for MQTT adapter)

Copied from original issue: EnMasseProject/mqtt-gateway#28

Currently, test reports and logs are printed to the travis log window. As we have many components, its hard to find where an error occured. To make our lives simpler, we should store build results and test reports in a place where its easier to navigate to each individual components report, and reduce the travis log verbosity to a minimum.

From @ppatierno on February 23, 2017 16:42

Copied from original issue: EnMasseProject/mqtt-lwt#1

In case the user wants to run the steps manually

From @lulf on September 16, 2016 13:13

Copied from original issue: EnMasseProject/openshift-configuration#13

There should be a non-secured e2e example for local development

From @lulf on March 3, 2017 10:40

The tls-mqtt-gateway is printing a log message every time the liveness probe tries to connect to it, presumably because the probe only does a TCP open and then close:

10:36:39.401 [vert.x-eventloop-thread-0] ERROR io.vertx.core.net.impl.NetServerBase - Client from origin /172.17.0.1:58716 failed to connect over ssl: java.nio.channels.ClosedChannelException

Copied from original issue: EnMasseProject/mqtt-gateway#24

As documented in https://github.com/EnMasseProject/enmasse/blob/master/documentation/address-model/resource-definitions.md#addressspace

From @ppatierno on January 9, 2017 13:53

The MQTT spec allows a client to connect using a 0 bytes client-id (in this case the clean session MUST be true).

If the Client supplies a zero-byte ClientId, the Client MUST also set CleanSession to 1 [MQTT-3.1.3-7]. If the Client supplies a zero-byte ClientId with CleanSession set to 0, the Server MUST respond to the CONNECT Packet with a CONNACK return code 0x02 (Identifier rejected) and then close the Network Connection.

The MQTT frontend needs to handle this scenario generating a random client-id for such clients in order to have an identifiers for needed AMQP links and messages.

Copied from original issue: EnMasseProject/mqtt-gateway#21

With the new addressing model, the getting started guides should be updated:

https://github.com/EnMasseProject/enmasse/blob/master/documentation/getting-started/openshift.md

From @ppatierno on April 26, 2017 15:22

While the addresses are propagating, it's fair to show the icon on the left side that explain that (the circles instead the checked green circle when propagation ends) but on the right side (i.e. messages in, ...) a "NaN" is showed that sounds like an error.
I would hide this part during propagation showing something like a "deploying ...." message until the system is ready and then showing the real info.

Copied from original issue: EnMasseProject/routilities#10

From @ppatierno on April 7, 2017 10:46

Consider following scenario on a topic ...

1. start a receiver, start a sender
2. sender sends 50 msgs -> receiver gets messages -> console shows 0 stored (ok, all messages flow to the unique active subscription)
3. stop receiver and sender
4. start sender, sends 50 msgs -> console shows 50 stored
5. sender sends 50 msgs -> console shows 100 stored
6. sender sends 50 msgs -> console shows 150 stored
7. start receiver (no messages got, ok)
8. sender sends 50 msgs -> receiver gets messages -> console shows 200 stored (?)
9. stop receiver
10. sender sends 50 msgs -> console shows 300 stored (!??, increased by 100)

Copied from original issue: EnMasseProject/routilities#5

From @lulf on June 9, 2017 12:30

The current health-check API created in the router agent has a vital limitation when used as a way to control router configuration: the source-of-truth for the router and addresses may not be up to date, so the health-check first and foremost checks that as far as the router agent knows, it has done its job.

For controlling that configuration is deployed through the router network, the router agent should provide an API for checking if an address has been configured correctly across a set of routers. The address and routers to check are provided as input to the API, and the output would be a list of router,status(ready=true/false, messages=[]) pairs .

This API can then be used by the address controller to update the address resource status, and hides the details of what is configured in the router from the address controller (and other interested parties).

Copied from original issue: EnMasseProject/ragent#15

When the images aren't available locally, it could happen that the AMQP - Kafka bridge image is download before the messaging one. In this case with the current liveness probe the bridge pod is restarted a lot of times because the messaging isn't up and running yet and it can't connect as client.
It's related to another problem in the bridge which start the HTTP server (for the liveness probe) after such connection is established (see strimzi/strimzi-kafka-bridge#73).
I'm going to move the HTTP server startup before the connection with messaging is established so that the pod can reply to the liveness probe but then provide another HTTP path (/ready ?) when the connection is established. Such path can be tested by the readiness probe.
In summary :

• the liveness probe is used just to check if the bridge is live even if it's not still connected to the messaging system
• the readiness probe works when the bridge is connected and is effectivily ready to receive

From @lulf on September 16, 2016 13:13

Copied from original issue: EnMasseProject/openshift-configuration#13

From @lulf on April 27, 2017 8:54

I'm getting a blank page when accessing the console with the latest image. See developer console debug.

It works with the 0.8.0 release, so I suspect that one of the changes in https://github.com/EnMasseProject/routilities/pull/9 is causing this since these are the only commits after the release.

@grs @ErnieAllen

Copied from original issue: EnMasseProject/routilities#11

From @ppatierno on June 1, 2017 15:5

When an MQTT subscriber connect to EnMasse, its related subscription doesn't show up as receiver in the addresses page.

Copied from original issue: EnMasseProject/routilities#27

Our use of templates prevents EnMasse from being deployed on a pure kubernetes cluster. Replacing templates with a third party resource (TPR) with the same schema, and adding a substitution mechanism in the address-controller should be sufficient.

Right now the console connects to the address-controller using the service environment. These environment variables are not available when they run in separate namespaces.

Work has already been done to prepare the console to use API_SERVER_HOSTNAME environment if available. The remaining work is to expose this environment variable so that it will be used by the console.

From @lulf on March 31, 2017 13:9

They still need to be unique, so current proposal is to use N characters from address and append the UUID. This is needed to avoid producing resource names larger than the kubernetes limit of 64 characters.

Copied from original issue: EnMasseProject/admin#7

See https://www.redhat.com/archives/enmasse/2017-July/msg00004.html for reference

To allow EnMasse to work with kubernetes, routes needs to be replaced with ingress. OpenShift can be configured to create routes automatically based on ingress resources.

If opening an AMQP link not mapping to a resource database in configserv, it eventually runs out of memory as the client reconnects. This should be investigated by running configserv and monitoring allocations.

From @lulf on November 28, 2016 7:32

Copied from original issue: EnMasseProject/artemis-image#8

From @ErnieAllen on May 3, 2017 16:6

If you filter the list of connections on the connections page, the values displayed are not updated.
When you clear the filter, the values are updated once but don't periodically update after that.

To get the values to update again, you need to switch to a new page and back or refresh the browser.

Copied from original issue: EnMasseProject/routilities#20

From @ppatierno on April 6, 2017 13:18

When a receiver attaches on a topic, the "addresses" tab reports Senders = 2, Receivers = 2 as values.

Copied from original issue: EnMasseProject/routilities#3

This allows users to specify instance and group id names that are not restricted to kubernetes label key restrictions.

See https://github.com/EnMasseProject/admin/issues/20

For OpenShift: https://docs.openshift.com/container-platform/3.5/dev_guide/secrets.html#service-serving-certificate-secrets

For kubernetes, we can add support for https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/

From @ppatierno on April 14, 2017 9:34

When a client drops the connection to the router, this sends a detach to the link on the MQTT LWT service with close=false. The handler for this event was added in the Vert.x Proton with this vert-x3/vertx-proton@8057708 so the need to add this handling to the MQTT LWT service.

Copied from original issue: EnMasseProject/mqtt-lwt#4

The LWT feature is not a really "MQTT related" but is having it on top of AMQP.

In the mqtt-lwt project I see :

• the Netty MQTT coded dependency is used just for the MQTT QoS enum, that could be removed (using an "ad hoc" enum or just 0,1 or 2) -> no dependencies on MQTT related projects anymore
• the only two AmqpWillMessage and AmqpPublishMessage classes involved use standard AMQP but add two annotations "x-opt-mqtt-qos" and "x-opt-retain-message". The first one could be not related to MQTT so we could remove the "-mqtt-" and in part it's decoded using the durable AMQP field (https://github.com/EnMasseProject/enmasse/blob/master/documentation/mqtt-over-amqp/docs/operations_connection.md#will-information-to-the-last-will-and-testament-service). The retain part is more MQTT specific (for the will message) even if you could not specify it having retain=false by default.

My instinct (but more thinking is needed) is to have an amqp-lwt service, so outside of the whole "mqtt" umbrella.

Wdyt ? @lulf @grs @rgodfrey

Adding an example for TLS using a certificates chain.

Create a root CA certificate.
Create a CA certificate signed by root CA.
Create server certificate signed by CA.
Using the CA certificate in the clients for authenticating the server.

From @ppatierno on May 24, 2017 12:26

MQTT QoS 1 passes even if a Paho "Timed out as no activity" is raised. Need to dig into it.

Copied from original issue: EnMasseProject/systemtests#12

From @ppatierno on May 3, 2017 1:49

If in the "Connections" page select a connection and click to see the details, when the info are updated (i.e. Messages In), the details automatically closes. I'm working on 0.9.0 release.

Copied from original issue: EnMasseProject/routilities#19

From @jcordes73 on December 8, 2016 19:2

Right now the queue/topic is exposing Jolokia at port 8161 (i.e. http://localhost:8161/jolokia). For OpenShift to be able to expose the Java console this needs to be changed and adapted by adding a port named jolokia like this

oc edit dc myqueue -o json
{
"name": "jolokia",
"containerPort": 8161,
"protocol": "TCP"
}

For this to work the jolokia port needs to be bound to 0.0.0.0.

Copied from original issue: EnMasseProject/openshift-configuration#22

The step related to project creation:

oc new-project enmasse

seems to be useless if the line related to run the deployment script has -p "enmasse".
Of course the script works but provides this line :

Using project "enmasse".
Error from server: project "enmasse" already exists

What do you think to remove the extra step of explicit creation of the "enmasse" project with new-project command outside of the script ?

Even because the script creates a project in any case with DEFAULT_OPENSHIFT_PROJECT name if -p option isn't used.

Document the new conceptual address model for EnMasse.

• Define address space, address and plans
• Document the 'standard' address space and its address types
• Document plans for the standard address space and its types
• Document resource definition of the model that is stored in Kubernetes

From @jcordes73 on December 8, 2016 18:44

Right now the broker is unauthenticated, i.e. you can access it without providing username and password when connecting from a client.

Copied from original issue: EnMasseProject/openshift-configuration#21

There are some messages classes which are duplicated between mqtt-gateway and mqtt-lwt projects. It could be better having an mqtt-common project for hosting such classes.

From @ppatierno on December 9, 2016 14:31

If a message is published from a native AMQP client it could have an AMQP value as body so using the AMQP type system (i.e. simple type but even list, map, ...). It could be delivered to an MQTT client. Do we have to consider a different encoding in this case ? (i.e. JSON as body of the MQTT message)

Copied from original issue: EnMasseProject/mqtt-gateway#10