krakendio / krakend-amqp

AMQP compatible backend for the KrakenD framework

krakend-amqp

AMQP compatible backend for the KrakenD framework

krakend-amqp's Issues

Not handling properly: channel/connection is not open

Hi everyone,

We're using KrakenD as our API Gateway and we're using krakend-amqp to directly publish in our RabbitMq cluster and it keeps losing the connection/channel, and it only recovers rebooting KrakenD.

Our configuration:

"extra_config": {
          "github.com/devopsfaith/krakend-amqp/produce": {
            "routing_key": "Route",
            "exchange": "search",
            "durable": true,
            "delete": false,
            "exclusive": false,
            "no_wait": true,
            "mandatory": true,
            "immediate": false
          }
        }

How to handle RPC and REST calls to the API gateway(Krakend)?

I'm trying to understand how to implement an api gateway that deals with microservices that expose rest and rpc endpoints.

• Expose REST endpoints to the client. Internally route to the REST or RPC microservices
• Expose REST and RPC endpoints to the client, route as necessary

We chose Krakend, I implemented the endpoints and configured rabbitmq, and it works, but I don't know how to give to the correct client from the queue. Can I give it back with some key?
@kpacha

first way
here i try set username with a queue and return the new username

{
      "endpoint": "/api/v1.0/setUserName/{reply_to}/{mess_id}/{route}",
      "method": "POST",
      "output_encoding": "json",
      "extra_config": {
        "github.com/devopsfaith/krakend/proxy": {
          "sequential": false
        }
      },
      "backend": [
        {
          "host": [
            "amqp://guest:guest@rabbitMQ:5672/"
          ],
          "url_pattern": "/",
          "encoding": "json",
          "sd": "static",
          "method": "GET",

          "disable_host_sanitize": true,
          "extra_config": {
            "github.com/devopsfaith/krakend-amqp/consume": {
              "name": "queue",
              "exchange": "exchange",
              "durable": true,
              "delete": false,
              "exclusive": false,
              "no_wait": true,
              "no_local": false,
              "routing_key": [
                "#"
              ],
              "prefetch_count": 10
            },

            "github.com/devopsfaith/krakend-amqp/produce": {
              "exchange": "exchange2",
              "durable": true,
              "delete": false,
              "exclusive": false,
              "no_wait": true,
              "mandatory": true,
              "immediate": false,
              "name": "queue2",

              "reply_to_key":"Reply_to",
              "msg_id_key":"Mess_id",
              "routing_key":"Route"
            }
          }

        }
      ]
    },

second way
producer
here i try set username with a queue

{
      "endpoint": "/api/v1.0/setUserName/{reply_to}/{mess_id}/{route}",
      "method": "POST",
      "headers_to_pass": ["Content-Type"],
      "backend": [
        {
          "host": [
            "amqp://guest:guest@rabbitMQ:5672/"
          ],
          "sd": "static",
          "encoding": "json",
          "disable_host_sanitize": true,
          "extra_config": {
            "github.com/devopsfaith/krakend-amqp/produce": {
              "name": "queue",
              "exchange": "exchange",
              "durable": true,
              "delete": false,
              "exclusive": false,
              "no_wait": true,
              "mandatory": true,
              "immediate": false,

              "reply_to_key":"Reply_to",
              "msg_id_key":"Mess_id",
              "routing_key":"Route"
            }
          }

        }
      ]
    },

consumer
here return the new username

    {
      "endpoint": "/api/v1.0/consumer/{queue_name}/{route}",
      "method": "GET",
      "backend": [
        {
          "host": [
            "amqp://guest:guest@rabbitMQ:5672/"
          ],
          "sd": "static",
          "encoding": "json",
          "disable_host_sanitize": true,
          "extra_config": {
            "github.com/devopsfaith/krakend-amqp/consume": {
              "name": "Queue_name",
              "exchange": "exchange2",
              "durable": true,
              "delete": false,
              "exclusive": false,
              "internal": false,
              "no_wait": true,
              "no_local": false,
              "routing_key": "Route",
              "prefetch_count": 10
            }
          }
        }
      ]
    }

thanks for help)

Customize the type of exchange

Hello,

I would like to be able to customize the type of exchange used. In hard, the value is "topic" and I would like to use "direct".

I made the modification on a local branch but I do not have the authorization to push it.
It would be enough simply to add a key "type" in the configuration :

# customer.go
# [...]
	err = ch.ExchangeDeclare(
		cfg.Exchange, // name
		cfg.Type,     // type
# [...]

# producer.go
# [...]
	err = ch.ExchangeDeclare(
		cfg.Exchange, // name
		cfg.Type,     // type
# [...]

# client.go
# [...]
type queueCfg struct {
	Name          string   `json:"name"`
	Type          string   `json:"type"`
# [...]

# client_test.go
# [...]
			consumerNamespace: map[string]interface{}{
				"name":           "queue-1",
				"type":           "topic",
				"exchange":       "some-exchange",
# [...]
			producerNamespace: map[string]interface{}{
				"name":      "queue-1",
				"type":      "topic",
				"exchange":  "some-exchange",

When do you think ?

PS : Excuse me for my English, I speak French.

Cannot produce message for exchange of type fanout.

Unfortunately I can't send messages to an Exchange configured as type fanout ... inequivalent arg 'type' for exchange 'user.events' in vhost '/': received 'topic' but current is 'fanout'. In the documentation I can't find a parameter where I can configure the type. Is fanout not supported or am I missing something?

krakend.json

{
  "version":3,
  "port":8080,
  "endpoints":[
    {
      "endpoint":"/api/user/event/signedUp",
      "method":"POST",
      "backend":[
        {
          "extra_config": {
            "backend/amqp/producer": {
              "name": "user.event.UserSignedUp",
              "exchange": "user.events",
              "durable": true,
              "delete": false,
              "no_wait": true,
              "no_local": false,
              "routing_key": "userSignedUp",
              "prefetch_count": 10,
              "mandatory": false,
              "immediate": false
            }
          },
          "host":[
            "amqp://guest:[email protected]:5672/"
          ],
          "disable_host_sanitize":true
        }
      ]
    }
  ]
}

KrakenD

krakend_ce_1 | 2022/04/25 14:15:52 KRAKEND ERROR: [ENDPOINT: /api/user/event/signedUp] Exception (504) Reason: "channel/connection is not open"
krakend_ce_1 | [GIN] 2022/04/25 - 14:15:52 | 500 |     293.667µs |      172.26.0.1 | POST     "/api/user/event/signedUp"

rabbitMq

rabbitmq | 2022-04-25 14:15:52.230932+00:00 [error] <0.4687.0> Channel error on connection <0.4678.0> (172.28.0.1:64784 -> 172.28.0.2:5672, vhost: '/', user: 'guest'), channel 1:
rabbitmq | 2022-04-25 14:15:52.230932+00:00 [error] <0.4687.0> operation exchange.declare caused a channel exception precondition_failed: inequivalent arg 'type' for exchange 'user.events' in vhost '/': received 'topic' but current is 'fanout'

Panic when host is not defined

When a host is not defined for and endpoint, the middleware will throw a panic since it's assuming all endpoints will have a host backend defined (and it should).

Config example:

{
  "version": 2,
  "extra_config": {},
  "timeout": "3000ms",
  "cache_ttl": "300s",
  "output_encoding": "json",
  "name": "testing",
  "endpoints": [
    {
      "endpoint": "/new2",
      "method": "GET",
      "extra_config": {},
      "output_encoding": "json",
      "concurrent_calls": 1,
      "backend": [
        {
          "url_pattern": "/v1/employees",
          "encoding": "json",
          "sd": "static",
          "extra_config": {}
        }
      ]
    }
  ]
}

Error:

[GIN-debug] GET    /new1                     --> github.com/devopsfaith/krakend/router/gin.CustomErrorEndpointHandler.func1 (3 handlers)
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/devopsfaith/krakend-amqp.backendFactory.initConsumer(0x24c9d80, 0xc0000aa040, 0x24d7080, 0xc000331a40, 0xc0001adbf0, 0xc0004a9530, 0xc000570790, 0x24c9d80, 0xc0000aa040, 0xc0003e1300, ...)
	/go/pkg/mod/github.com/devopsfaith/[email protected]/consumer.go:28 +0x159d
github.com/devopsfaith/krakend-amqp.backendFactory.New(0x24c9d80, 0xc0000aa040, 0x24d7080, 0xc000331a40, 0xc0001adbf0, 0xc0004a9530, 0xc000570790, 0xc0003e1300, 0x100)
	/go/pkg/mod/github.com/devopsfaith/[email protected]/client.go:35 +0x7a

Producer: routing_key works only for URL parameters

Hello,
Following the documentation here https://www.krakend.io/docs/backends/amqp-producer/#routing_key the routing key is expected to work with static values.

Unfortunately, this functionality is not supported by the code.

I am willing to open a Pull Request to make the code supporting this. Are you ok with including this functionality?

@kpacha @alombarte is there any alternative to make use of static routing values?

Add ability to override exchange type for the producer, because it is hardcoded to `topic`

Add ability to override exchange type for the producer, because it is hardcoded to topic

Unsupported Protocol Scheme

Hi everyone! first of all, awesome products you've been building.
So we're thinking on using KrakenD as our API Gateway, and since we have a message-driven API built on top of RabbitMQ for our internal communication between services, we've been looking into using this plugin to allow external parties to call a rest endpoint and Kraken to transport to AMQP and so on..
Everything looks good, we've installed Kraken as a standalone daemon and the plugin works fine, but as soon as we put together a Docker image (we have everything containerized) we started hitting this error with the exact same configurations for everything:

gateway_1     | [GIN] 2021/01/13 - 13:55:38 | 500 |     170.544µs |    192.168.32.1 | GET      /testpub2
gateway_1     | Error #01: Get amqp://minpoc:***@messagebus:5672/: unsupported protocol scheme "amqp"

Running kraken with --debug throws a more descriptive error. Still unsure what's causing it though

[GIN] 2021/01/13 - 14:06:17 | 500 |      66.303µs |    192.168.32.1 | GET      /testpub2
Error #01: Exception (504) Reason: "channel/connection is not open"

I haven't been able to find anything anywhere about this, any ideas??

Thanks a lot!

amqp producer: static routing_key is ignored

Hello,

I declare this endpoint, with an amqp producer backend:

      "endpoint": "/chamallows",
      "method": "POST",
      "input_headers": ["*", "Cookie"],
      "backend": [
        {
          "host": ["amqp://guest:guest@rabbitmq:5672"],
          "disable_host_sanitize": true,
          "url_pattern": "/",
          "extra_config": {
            "backend/amqp/producer": {
              "name": "",
              "exchange": "sugar-entrance",
              "durable": true,
              "delete": false,
              "no_wait": false,
              "no_local": false,
              "routing_key": "haribo",
              "mandatory": false,
              "immediate": false
            }
          }
        }
      ]

The routing_key: "haribo" isn't transmitted to rabbitmq. And it seems that only routing_key from url parameters (with capitalization) is working. But in my case I can't use this feature. I don't see this distinctive characteristic in the doc

And I think the reason of that come from this line: https://github.com/krakendio/krakend-amqp/blob/1948b9d0c1465fddb67a88dfe715816666e1a7f6/producer.go#L124

Am I missing something? Is this behavior intentional? If not, is it a problem that can be quickly solved on your side?

Thanks for your work

Support for RPC calls

Hi everyone! Are there any plans to have this plugin also able to make RPC calls?
As our current PoC is being develop, we're looking into a way for having a REST API exposed to clients (eg: web app or third party devs) while having all our backends behind rabbitmq.
In order to do so, and because the plugin right now only supports fire n forget, we're looking into enabling RPC inside this plugin so we can leverage all Rabbit's flavors (such as autoscaling and so on) instead of plugin directly our services to kraken.
Hope this makes sense, i'll start working on a fork of this repo, but wanted to first check out if you guys had any plans for this.

Thanks again!